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Abstract 

o 

rsq . Let # be a uniformly distributed random A;-SAT formula with n variables and m clauses. Non-constructive 

arguments show that $ is satisfiable for clause/variable ratios m/n < r^ ~ 2 In 2 with high probability. Yet 
^ , no efficient algorithm is know to find a satisfying assignment for densities as low as m/n ^ r^ ■ ln(fc)/fc with a 

non-vanishing probability. In fact, the density ra/n ~ rk ■ \n{k)/k seems to form a barrier for a broad class of 

QQ ' local search algorithms. One of the very few algorithms that plausibly seemed capable of breaking this barrier is 

a message passing algorithm called belief propagation guided decimation. It was put forward on the basis of deep 
but non-rigorous statistical mechanics considerations. Experiments conducted for fc = 3, 4, 5 suggested that the 
algorithm might succeed for densities very close to r^. Furnishing the first rigorous analysis of BP decimation, 

r) , the present paper shows that the algorithm /fl//.s to find a satisfying assignment already for m/n > p ■ r^/k, for 

a constant p > Q independent of k. 
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1 Introduction and results 

Let A: > 3 and ji > 1 be integers, let r > be a real, and set m = \rn] . Let $ = $fe (n, rn) be a prepositional 
formula obtained by choosing a set of m clauses of length k over the variables xi, . . . ,Xn uniformly at random. 
For k, r fixed we say that $ has some property V with high probability ('w.h.p.') if lim„^oo P [^ G V] = I. 

Non-constructive arguments show that w.h.p. $ is satisfiable if r < r^ = 2^^ In 2 — fc, and unsatisfiable if 
r > 2'^ In 2 |[3]|5l . However, in spite of several decades of extensive research in the CS community, both theoretical 
and experimental, no algorithm has emerged to find a satisfying assignment for densities r anywhere close to r^ 
in polynomial time w.h.p. More precisely, the best current polynomial time algorithm, designed specifically to 
solve random fc-CNFs, fails for r > Vk ■ ^^ w.h.p. lfT2ll . Furthermore, a great many algorithms are known to 
fail for asymptotically even smaller densities r > Vk ■ ^, where p > is a constant (independent of k) that 
depends on the sophistication of the algorithm. Examples include simple linear-time algorithms such as Unit 
Clause (p = e/2) iflOl or Shortest Clause (p — 1.817) ifTSl . as well as a wide range of DPLL-type algorithms 
(p = 11/4) |[T]. More generally, in [2] we provided evidence that the density r — r^ ■ -^p may form a barrier for 
a broad class of local search algorithms. 

Against the background of these negative results, it came as a surprise when experiments indicated that certain 
message passing algorithms solve random /c-CNFs in polynomial time for densities r very close to the conjectured 
thresholds for the existence of satisfying assignments, at least for fc = 3,4, 5 ||8][19]|2T|. These algorithms, namely 
Belief Propagation guided decimation and Survey Propagation guided decimation, were put forward on the basis 
of very insightful but highly non-rigorous statistical mechanics considerations |[8]|20l. Indeed, these message 
passing algorithms have come to be considered the only plausible 'candidates' for overcoming the r^ • ^^ barrier 
from 121. Therefore, since these algorithms were suggested, one of the most important challenges in this area has 
been to come up with a rigorous analysis Q. 

The present paper contributes the first rigorous analysis of BeUef Propagation guided decimation for random 
fc-SAT and, in fact, for any random constraint satisfaction problem. We establish a negative result: for general 
k the BP decimation algorithm BPdec fails to find a satisfying assignment w.h.p. for densities r > r^ • | for a 
certain constant p > 0. In other words, for general k BPdec does not outperform other, simpler algorithms (such 
as the one from fV2\). 

Stating the result precisely requires a little care, because it involves two levels of randomness; the choice of 
the random formula #, and the 'coin tosses' of the randomized algorithm BPdec. For a (fixed, non-random) 
fc-CNF $ let success($) denote the probability that BPdec(<I>) outputs a satisfying assignment (where, of course, 
'probability' refers to the coin tosses of the algorithm). Then, if we apply BPdec to the random fc-CNF #, the 
success probability success(#) becomes a random variable. Recall that $ is unsatisfiable for r > 2^ In 2 w.h.p. 

Theorem 1.1 There is a constant po > such that for any fc, r satisfying 

po-2''/k<r <2''\n2 (1) 

we have success(#) < exp(— 51(?t,)) w.h.p. 

Theorem I 1.11 contrasts with the rather promising experimental results on the performance of BPdec($). But 
these results were obtained for 'small' k ~ 3,4,5. Already for fc = 10 experiments are difficult to conduct, 
because the relevant density r scales exponentially in fc. On the other hand, the constant po in Theorem 11.11 
implies a lower bound on fc, as for small fc the range ([T]i may just be empty. (Since the analysis is intricate as is, 
no attempt has been made to compute or optimize the precise value of po.) 

An objection to a negative result like Theorem I 1 . 1 I could be that it might hinge on a small detail of the algorithm 
that could easily be fixed. However, in the sequel we will see that our analysis implies that in the regime ([T]i the 
key hypothesis on which BPdec is based is untrue. 

The BeUef Propagation guided decimation algorithm. Q Fix a satisfiable fc-CNF $ on the variables V = 
{xi, . . . ,Xn}. The algorithm BPdec is an attempt at implementing the following thought experiment, which 
yields an assignment ct : V^ — > {0, 1} (with representing 'false' and 1 representing 'true'). 

Experiment 1.2 Input: A satisfiable k-CNF $. Result: An assignment a -.V ^f {0, 1}. 



'in addition to tlie following discussion, more details on Belief/Survey propagation and the statistical mechanics perspective on random 
fc-SAT can be found in Appendix IaI 



0. /.ef$o = $. 

1. Fort = 1,. . . ,n do 

2. Compute the fraction M^t (^t-i) of all satisfying assignments of $4-1 In which the variable xt 
takes the value 1. 

3. Asslgna{xt) = 1 with probability Mx,{^t-i), and leta{xt) = otherwise. 

4. Obtain the formula ^t from^t-i by substituting the value (T{xt) forxt and simplifying (i.e., delete 
all clauses that got satisfied, and omit xt from all other clauses). 

5. Return the assignment a. 

A moment's reflection reveals that the above experiment not only produces a satisfying assignment, but that its 
(random) outcome is in fact uniformly distributed over the set of all satisfying assignments of $. The obvious 
obstacle to actually implementing this experiment is the computation of the marginal probability M^^ ($t-i ) that 
Xt takes the value 'true' in a random satisfying assignment of $4-1. Indeed, computing A/^j($t_i) is #P-hard 
in the worst case. 

Yet, under what conditions could we hope to compute (or 'approximate') the marginals Mj:{^t-i) in the 
'decimated' formula $t-i? Clearly, the AIx{^t-i) are influenced by 'local' effects. For instance, if x occurs in a 
unit clause a of $f_i, i.e., a clause whose other fc — 1 variables have been assigned already without satisfying a, 
then X must be assigned so as to satisfy a. Hence, if x appears in a positively, then Mx{^t-i) = 1, and otherwise 
Ma;($(_i) — 0. Similarly, if x occurs only positively in $f_i, then Mx{^t~i) > 1/2. More intricately, if 
X occurs in a clause a that contains another variable y that appears is a unit clause b, then this will affect the 
marginal of x. 

The key hypothesis underlying BPdec is that in random formulas such local effects determine the marginals 
Mx{^t-i) asymptotically. To define 'local' precisely, we need a metric on the variables/clauses. This metric is 
the one induced by the /acfor graph G = G{^t-i) of $t-i, which is a bipartite graph whose vertices are the 
variables Vt = {xt, ■ . ■ , Xn} and the clauses of $t_i. Each clause is adjacent to the variables that occur in it. 
For an integer w > 1 let N'^{xt) signify the set of all vertices of G that have distance at most 2w from Xf. Then 
the induced subgraph G [N'^{xt)] corresponds to the sub-formula of $t_i obtained by removing all clauses and 
variables at distance more than 2a; from xt. Note that all vertices at distance precisely 2a; are variables, so that any 
satisfying assignment of $ induces a satisfying assignment of the sub-formula. Let us denote by Mx^i^t-i,'^) 
the marginal probability that xt takes the value 1 in a random satisfying assignment of this sub-formula. 

Of course, for a 'worst-case' formula $ the 'local' marginals Mx^ ($t-i, w) may be just as difficult to compute 
as the overall marginals Mx-,{^t-i) themselves. Therefore, BPdec employs an efficient heuristic called Belief 
Propagation ('BP'), which yields certain values pxti^t-i,^) G [0,1]; we will state this heuristic below. If 
the induced subgraph G[N'^{xt)] is a tree, then indeed ij,j-^{^t-i,u!) = A/a;j($4_i,a;). Moreover, standard 
arguments show that in a random formula # actually G [N'^{xt)] is a tree w.h.p. so long as a; = o(lnn). More 
generally, in order to obtain an efficient algorithm it would be sufficient for the BP outcomes pxti^t-i,^) to 
approximate the true overall marginals Mx^i^t-i) well for some (say, polynomially computable, polynomially 
bounded) function a; = uj{7i) > 1. This leads to the following hypothesis underpinning BPdec (cf. Il20l ). 

Hypothesis 1.3 With probability 1 — o(l) over the choice of ^ and the random decisions in Experiment U .2\ the 
following holds for all 1 < t < n. 

i. For any e > there is lo ~ i-^{s., k, r) such that |Aia.j($f_i) — Mxt{^t-i,'-^)\ < C- 

//. There is a function Lo — uj{n) > 1 such that M^ti^t-i) — IJ-xti^t-i,^) + o{l) w.h.p. 

Hvpothesis 1 1 . 3l motivates the following algorithm ll24l . which is called BP_^uided decimation because it com 
bines BP (Step 2) with a procedure to 'decimate' the formula (Steps 3 and 4) 



r 



Algorithm 1.4 BPdec(<l>) 

Input: A fc-CNF $ on F = {a;i, . . . , x„}. Output: An assignment ct : F — > {0, 1}. 

0. Let $0 = $. 

1. Fori = 1, ... ,n do 

2. Use BP to compute ^i:t ($4-1, a;). 

3. Assign o{xt) = 1 with probability ^a;^ ($4-1,0;), and letCT(a::4) = otherwise. 



^BPdec is the simplest, but arguably also the most natural version of a 'BP decimation' algorithm. Appendix IaI contains a discussion of 
other version of BP decimation. 



4. Obtain the formula <E>t from <I>f _i by substituting the value a{xt) for xt and simplifying. 

5. Return the assignment a. 

Remark 1.5 The statement of Theorem U .l\ holds regardless of the function uj ~ Lo{n) > chosen in Step 2. 

Although, strictly speaking. Hypothesis 1 1 . 3l provides neither a necessary nor a sufficient condition for BPdec 
to succeed on random fc-CNFs w.h.p., the hypothesis inspired the algorithm. In fact, it plays an important role in 
the statistical mechanics work on random fc-SAT generally, see Section|2]below. The proof of Theorem ! 1 . 1 l entails 

Corollary 1.6 Both statements of Hvpothesis \1.3\ are false for k, r satisfying (Q. 

We finish this section by briefly defining Belief Propagation for fc-SAT; for a detailed derivation we refer the 
reader to |I8]|26]. Ultimately, we need to define the value fi^t (*&t-ij w) for Step 2 of BPdec. 

Let N{v) denote the neighborhood of a vertex v of the factor graph G(<l>t_i). For a variable x G Vt and a 
clause a G N{x) we will denote the ordered pair {x, a) by x — > a. Similarly, a — > a; stands for the pair (a, x). 
Furthermore, we let sign(a;, a) = 1 if a; occurs in a positively, and sign(a;, a) = — 1 otherwise. 

The message space M{^t-i) is the set of aU tuples {f^x-ya{C))xeVt.aeN{x),c&{o,i} such that //a;_>a(C) ^ [0, 1] 
and Hx^aiO) + /^a:-i-a(l) = 1 for all x, a, (. For p E il/($) we define 

J 1 ifC = (l + sign(x,a))/2, 

[ ^ [lyeN{a)\{x}lh^ay 2 J otncrwise. 

Furthermore, we define the belief propagation operator BP as follows: for any p e M($t_i) we define BP{p) E 
M($4_i) by letting 

(BP(M)).-.a(c) = J. n..^(.)\MM.^.(c) ^3^ 

i.heN{x)\{a} f-b^xi^) + ].[beN{x)\{a} f-b^x^i-) 

unless the denominator equals zero, in which case (BP{p))x->a{0 = 5- 

Finally, the values fixi^t,!^) that are used in step 3 of BPdec are defined as follows. Let p[0] = ^ ■ 1 E 
M($t_i) be the vector with all entries equal to i. Moreover, define inductively p[£] — BP(/i [£ — 1]) for 
1<£<L0. Then 

/* , UbeN{x)l^b^M)[^] 

p^{<^t^i,oj) = — .rl^ r 1 I TT TTVn foranyxSVt, (4) 

UbGN{x) A«b->x(0) H + UbeN{x) M&^x(l) H 

unless the denominator is zero, in which case we set px{^t^i,io) = i. 

The intuition here is that the px-^a{C) ^6 'messages' from a variable x to the clauses a in which x occurs, 
indicating how likely x were to take the value ( if clause a were removed from the formula. Based on these, ^ 
yields messages pa^xiC) from clauses a to variables x, indicating the probability that a is satisfied if x takes 
the value C, and all other variables y G N{a) \ {x} are assigned independently to either value ^ G {0, 1} with 
probability fiy^aiO- The BP operator (O then uses these messages pa^x in order to 'update' the messages from 
variables to clauses. More precisely, for each x and a € N{x) the new messages (BP{fi))x^a{C) ^re computed 
under the hypothesis that all other clauses b E N{x) \ {a} are satisfied with probabilities n^^xiO independently 
if X takes the value (. Finally, the difference between Q and (JUi is that the latter product runs over all clauses 
b E N{x). An inductive proof shows that, if for a variable x the subgraph G [N'^{x)] of the factor graph is a tree, 
then in fact /x^($t,w) =M^($t,w) JH). 

2 Related work 

2.1 The statistical mechanics perspective. 

The statistical mechanics work on random fc-SAT is based on the 'cavity method', an ingenious, albeit highly 
non-rigorous, technique from the theory of spin glasses 11211 . This actually is an analytic technique that yields, 
among other things, a conjecture on the precise location of the thresholds for satisfiability for any fc. BPdec can 
be viewed as an attempt at turning (a part of) the cavity method into an efficient algorithm 



However, even the cavity method does not yield direct, quantitative estimates as to the regime of clause den- 
sities r for which BPdec($) succeeds w.h.p. The obstacle is the decimation step of BPdec, i.e., the fact that 
BPdec eliminates one variable in each iteration, and then repeats the Belief Propagation computation on the re- 
duced formula. In fact, tracing the BP decimation process directly (which is the approach taken in the present 
paper) has been attributed as "extremely challenging" |(24l|. 

In an important paper |20| the cavity method was used to determine the density re up to which statements i. 
and ii. in Hypothesis [O] hold for i = 1, i.e., in the undecimated random formula $. This density Tc is called 
the condensation point. The cavity method yields Tc = 2*^ In 2 — | In 2, which is very close to the conjectured 
satisfiability threshold 2^^ ln2 — ^(1 + In 2). Indeed, ||20| goes on to forecast that BPdec "should perform well 
up to the condensation point" re- 

The issue with this prognosis (which would not sit well with Theorem II . 1 b is that it disregards the effect 
of the decimation process, as pointed out recently in ll27l . That paper extends the cavity method to analyze 
Experiment ll.2l i.e., the idealized procedure that BPdec was designed to implement. This (non-rigorous) analysis 
suggests that by decimating, i.e., by assigning and eliminating variables, for t sufficiently close to n the reduced 
formula ^t exhibits a 'condensation' phenomenon even at densities r far below Vc- Quantitatively, the analysis 
from Il27l suggests that Hvpothesis 11.31 is invalid for r > c • 2^/k. Yet Hvpothesis 11.31 is neither necessary 
nor sufficient for the success of BPdec($). Indeed, the paper ll27l . which supersedes an earlier non-rigorous 
study |24j, mentions a rigorous analysis of BPdec expUcitly as an open problem. 

2.2 Rigorous work. 

Belief Propagation (i.e., the message passing procedure to 'approximate' the marginals of a probability distribution 
described by a factor graph) is a fairly generic technique [1261 . If the factor graph is acyclic, then Belief Propagation 
provably computes the correct marginals, which is generally not true in the presence of cycles. Nevertheless, BP is 
often applied as a heuristic to factor graphs with cycles anyway in the hope that it may yield useful approximations. 

In spite of BP's practical success (and popularity), rigorous analyses are scarce. A few exist in the context of 
LDPC decoding Il22ll28l . We also analyzed BP for graph 3-coloring llT3l on a certain class of expander graphs. 
A further related result deals with another, conceptually far simpler message passing algorithm called Warning 
Propagation ('WP') on certain random 3-CNFs ("planted model") |[T4l . 

The big conceptual difference between the present and the previous work is that the algorithm BPdec studied 
here combines BP with decimation (i.e., the algorithm assigns one variable at a time and reruns BP on the reduced 
formula). The present paper furnishes the first analysis of this kind of algorithm. By comparison, previous analyses 
deal with algorithms that use BP in a 'one shot' fashion, i.e., the 'marginals' obtained via BP are used directly 
to assign all variables at once lfT3l l22l l28l . Roughly speaking, this approach works when the problem instances 
are somewhat over-constrained so that there is (essentially) a unique solution. By contrast, random /c-CNFs with 
densities r < r^, i.e., below the fc-SAT threshold have exponentially many satisfying assignments whose typical 
pairwise distance is close to n/2 w.h.p. Q. 

The study of the BP marginals on the undecimated random formula $ is somewhat related to the so-called 
reconstruction problem. This problem has been studied on 'symmetric' random CSPs, which include problems 
such as (hyper)graph coloring, but not fc-SAT |23 1. The proofs are based on indirect arguments (closely related to 
the second moment method), which do not seem to extend to an analysis of BPdec. 

Finally, the paper ||251 establishes r ^ ^-^^ as the threshold for Gibbs uniqueness in random fc-SAT. The 
proof is via studying the correlation decay of formulas whose factor graphs are random trees. The result implies 
that up to density r ^ ^^ it is true that Mx{^) = /ii^(*, i^) + o(l) if 1 < i:j < In?! w.h.p. That is, BP does 
actually approximate the true marginals well on the original, undecimated formula. However, this result does not 
extend to an analysis of the decimation process either Note that the density r ^ ^-^§^ to which 1251 applies is far 
below the f7(2'°/fc) addressed in Theorem ll.il 

3 The probabilistic framework for analyzing BPdec 

The single most important technique for analyzing algorithms on the random fc-CNF $ is the method of deferred 
decisions. Where it applies, the dynamics of the algorithm can typically be traced tightly via differential equations, 
martingales, or Markov chains. Virtually all of the previous analyses of algorithms for random fc-SAT are based 
on this approach ||6] |9] [JO] [TTl [12] [15] US] [ISI . Unfortunately, the 'deferred decisions' technique is limited to very 
simple, 'shortsighted' algorithms that decide upon the value of a variable x on the basis of the clauses/variables at 



distance, say, one or two from x in the factor graph ||6l. By contrast, in order to assign the ith variable xt, BPdec 
explores clauses at distance up to 2uj from xt, where (potentially) oj = uin) — > oo. This renders a 'deferred 
decisions' approach hopeless. 

Therefore, to prove Theorem II. II we need a fundamentally different strategy. In the present section we set up 
the probabilistic framework for the analysis. We will basically reduce the analysis of BPdec to the problem of 
analyzing the BP operator on the formula that is obtained from the random $ by substituting 'true' for the first 
t variables xi, . . . , Xt (Theorem 13.31 blow) . In the next section we will show that this decimated formula enjoys 
a few simple quasirandomness properties with probability extremely close to one. Finally, in Section |5] we will 
show that these quasirandomness properties suffice in order to trace the dynamics of the BP computation. 

Applied to a fc-CNF ^ onV — {xi, . . . , x„}, BPdec yields an assignment a -.V ^^ {0, 1} (that may or may 
not be satisfying). This assignment is random (even if $ is not), because BPdec itself is randomized. Hence, for 
any fixed $ running BPdec($) induces a probability distribution /3$ on {0, 1} . Let S'($) denote the set of all 
satisfying assignments of $. Then the 'success probability' of BPdec on$ isjust success($) = /3$(S'($)). Thus, 
to establish Theorem 1 1 . 1 1 we need to show that in random formulas success(#) = /3#(5(#)) is exponentially 
small w.h.p. To this end, we are going to prove that the measure /3,^ is 'close' to the uniform distribution on 
{0, 1} w.h.p., of which 5($) constitutes only an exponentially small fraction. 

For the sake of the analysis, we consider a slightly modified version of BPdec. While the original BPdec 
assigns the variables in the natural order xi, . . . , x-„, the modified version PermBPdec chooses a permutation tt 
of [n] uniformly at random and assigns the variables in the order a;7r(i)5 • ■ • i a;^(„). Let /3$ denote the probability 
distribution induced on {0, 1} by PermBPdec($). Because the uniform distribution over /c-CNFs is invariant 
under permutations of the variables, we obtain 

Fact 3.1 ///3#(S'(*)) < cxp(-ri(7i)) w.h.p., then success(*) ^ (i^{S{^)) < cxp(-fi(n)) w.h.p. 

Let <!> be a fc-CNF and let S > 0. Given a permutation n and a partial assignment a : {x^^^s) '■ s <t^ ^ {0, 1}, 
we let ^t,TT,a denote the formula obtained from $ by substituting the values cr(x^(s)) for the variables x^r^s) for 
1 < s < t and simplifying. For a number 6 > and an index I > twe say that a;^(/) is {6, t)-biased if the result 
Ma;„(n (^t,7r,cr, w) of the BP Computation on ^t.ir.a differs from ^ by more than 5, i.e., 

|M:^,(„($t,7r,<T,w)-l/2| > 5. 

Moreover, the triple (<I>, tt, a) is ((5, t)-balanced if no more than 5{n — t) variables are {S, t)-biased. 

Let TT be the permutation chosen by PermBPdec($), and let a be the partial assignment constructed in the 
first t steps. The variable a;^(t^i) is uniformly distributed over the set V \ {xt,(^s) : s < t} of currently unassigned 
variables. Hence, if ($,7r,cr) is (5, i)-balanced, then the probability that XTr{t+i) is ((5, t)-biased is bounded by 
S. (This conclusion was the purpose of decimating the variables in a random order) Furthermore, given that 
2^ir(t+i) is not ((5, t) -biased, the probability that PermBPdec will assigned it the value 'true' lies in the interval 
[i — (5, i + 5] . Consequently, 



- -P [cr(x^(t+i)) = l|($,7r,cr)is ((5,i)-balanced] 



< 26. 



(5) 



Thus, the smaller 5, the closer cr(a;^(t+i)) comes to being uniformly distributed. Hence, if ((5, i)-balancedness 
holds for all t with a 'small' S, then /3$ will be close to the uniform distribution on {0, 1} . 
To put this observation to work, we define 



St = exp(-c(l - t/n)k) and T = (1 - r/2'')n, 



(6) 



where c > is a small absolute constant (independent of fc, r, t, n). Furthermore, for ^ > we say that $ is 
(t, ^)-uniform if the total number of permutations n and assignments a g {0, 1} such that ($, tt, a) is not 

((5t, t)-balanced is bounded by 2"n! • exp — ^n — 10 J2s=i ^s ■ Proceeding by induction on t, we can use Q to 
relate the distribution /3^ to the uniform distribution on {0, 1} for formulas that are {t, ^)-uniform. 

Proposition 3.2 Suppose that $ is {t, ^)-unifonnfor all < t <T. Then 



;3*(f)<P-Cxp 



T 



cxp(-^n/2) for any £ C {0, 1}^ 



(7) 



Proposition 13.21 reduces the proof of Theorem 1 1 . 1 1 to showing that $ is (i, ^)-uniform. The following theorem 
provides the key step. 

Theorem 3.3 There is a constant po > such that for any k, r satisfying 2^ p^/k < r < 2^^ lrL2 there is ^ = 
^{k, r) > so that for n large enough the following holds. Fix any permutation -k of [n] and any assignment 
(7 G {0, 1} . Then for any < t < T we have 



P [(#, TT, a) is (dt, t)-balanced\ > 1 — exp — 2^n — 10X]s=i ^^ 



(8) 



Theorem 13.31 shows that for a fixed pair (7r,cr) the triple ($,7r, cr) is likely to be balanced. A simple first 
moment argument then yields a bound on the number of pairs (tt, a) so that ($, tt, a) fails to be balanced. This 
implies that $ is likely to be (t, ^)-uniform. 

Corollary 3.4 In the notation ofTheorem \3.3\ P [Vi < T : $ is {t, S,)-unifomi\ > 1 — cxp(— ^7i). 

Combining Proposition l3.2l and Corollarv l3.4l we see that O holds for a random formula 4> w.h.p. In particular, 
we can apply (|7]) to the set 8 = S{'^) of satisfying assignments. A simple first moment calculation shows that 
|£-| < 2(i+o(i))'i(i _ 2"'=)" < cxp[n(ln2 - r2-^ + o(l))] w.h.p. Plugging this bound into ©, recalling that 
T = (1 — r/2^)n, and observing X]t=i ^t ~ n/{ckexp{ckr/2'')) yields Theorem ll.il 

4 The quasirandomness property 



The goal in this section is to establish Theorem l3.3l which states that for any fixed permutation tt and any fixed 
assignment a the triple ($, tt, ct) is {6t , t)-balanced with probability very close to one. By the symmetry properties 
of the uniformly distribution over /c-CNFs, we may assume without loss of generality that tt = id is the identity 
permutation, and that cr = 1 is the all-true assignment. Then the decimated formula 4>t = ^t.iT,cr is obtained from 
$ just by substituting the value 'true' for xi, . . . ,xt and simplifying. In this section we will exhibit a few simple 
quasirandomness properties that $t is very likely to possess. In Section [5] we will show that these properties 
suffice to trace the BP operator. 

To state the quasirandomness properties, fix a fc-CNF $. Let $t = $t id,i denote the CNF obtained from $ by 
substituting 'true' for xi, . . . ,xt and simplifying (1 < t < n). Let Vt+i = {xt+i, ■ ■ ■ , Xn} be the set of variables 
of <l>t. As before, we will denote the factor graph of <l>t by G = G'($t), and the neighborhood of a vertex v by 
N{v). We call a clause a of $ redundant if there is another clause b such that \N {a) D N {b)\ > 2. Let 

= 1- t/n 

be the fraction of unassigned variables. Moreover, for a variable x E Vt and a set Q C Vt let 

N<i{x,Q) = {beN{x):\N{x)r\Q\{x}\<lAO.Wk<\N{b)\<lO0k}. (9) 



Thus, iV<i(a;, Q) is the set of all clauses that contain x (which may or may not be in Q) and at most one other 
variable from Q. In addition, there is a condition on the length \N{b)\ of the clause b in the decimated formula 
^f Observe that having assigned the first t variables, we should 'expect' the average clause length to be 9k. For 
a linear map A : R^* -^ R^* let ||A||p signify the norm ||A||p = max^gR^vt^jg} l/^i ■ 

Definition 4.1 Let S > 0. We say that $ is (5, t)-quasirandom i/Q0-Q4 in Figure\T\are satisfied. 

Condition QO simply bounds the number of redundant clauses and the number of variables of very high degree; 
it well-known to hold for random fc-CNFs w.h.p. Ql provides a bound on the 'weight' of clauses in which variables 
X <E Vt+i typically occur, where the weight of a clause bis 2^l^'^''^l. Moreover, Q2 imposes that there is no small 
set Q for which the total weight of the clauses touching that set is very big. In addition, Q2 (essentially) requires 
that for most variables x the weights of the clauses where x occurs positively/negatively should approximately 
cancel out. Q3 provides a bound on the lengths of clauses that contain many variables from a small set Q. 

The most important condition is Q4. Think of a Vt+i x Vf+i -matrix representation of the formula where 
the entry of a pair (x, y) of variables is the sum of the signed weights 2~^('''sign(x, b)sign{y, b) over clauses 
b e N{x) n N{y). This can be considered a weighted, signed adjacency matrix of $t. Then Q4 requires that this 
matrix has low discrepancy (i.e., it is 'flat'), modulo the clauses that contain more than one variable from some 
small 'exceptional' set Q. 



QO. $ has only o(n) redundant clauses. Moreover, no more than o{n) variables occur in more than In n clauses of^. 
Ql. No more than 10~^S{n — t) variables occur in clauses of length less than 9k/10 or greater than 109k. Moreover, 
there are at most 10^^i5(n — t) variables x G Vt+i such that 

(^fc)^'5-E.e^(.)2-l^('')l>l. 
Q2. IfQC Vt+i has size |Q| < 5{n — t), then there are no more than 10^'*(5(n — t) variables x such that either 



Y, 2-I^WI > (e/fc)^^, or 

beN{x):\N{b)nQ\{x}\ = l 



y^ 2\Nib)nQ\{x}\-\N{b)\ ^ _^^ ^^ 

beN{x):\N{b}nQ\{x}\>i 



_s_ 

Ok' 



E 



sign(a;, b) 



2lJV(!))| 



> 



1000 



bi£N<i{x,Q) 

Q3. For any 0.01 < z < 1 and any set Q C Vt+i of size 0.016{n — t) < \Q\ < I006{n — t) we have 

b:\N(b)nQ\>z\N{b)\ 

Q4. For any set Q C Vt+i of size IQI < 10S{n — t) the linear operator 



Aq : R^*+i ^ R^'+\ r 



^ ^ 2-I^WIsign(x,6)sign(y,6)r, 

beN^il^x,Q)y(^N(b)\{x] 



xeVt^ 



has norm ||Aq|||^ < S'^{n — t). 

Figure 1: The conditions for Definition l4.1 



Proposition 4.2 There exists a constant po > such that for any k, r satisfying po ■ 2^ jk < r < 2*^ In 2 there is 
C = ^e^i r) > so that for n large and St, T as in (O we have 



P [$ is {St, t)-quasirandom] > 1 — exp —10 [(.n + X]s<t ^s 



for any I < t < T. 



Though non-trivial, the proof of Proposition 14.21 is based on standard arguments. Together with the following 
theorem, which we will estabUsh in Section|5] Proposition |42] yields Theorem l3.3l 

Theorem 4.3 There is po > such that for any k, r satisfying po ■ 2^ /k < r < 2*^ In 2 and n sufficiently large 
the following is true. Let ^ be a k-CNF with n variables and m clauses that is (St,t)-quasirandofnfor some 
1 < t < T = (1 - r/2^)n. Then ($, id, 1) is {St,t)-balanced. 



5 Tracing the Belief Propagation operator 

Throughout this section, we keep the notation from Section |4] and the assumptions of Theorem 14.51 Assum- 
ing that $ is ((5f, t)-quasirandom, we aim at showing that in the formula $t obtained by substituting the value 
'true' for the first t variables xi, . . . ,Xt the values p^,, (*&t, w) computed by belief propagation are in the interval 
[^ — (5(, ^ + (St] , for all but at most 5t{n — t) indices t < s < n. 

To establish this, we are going to trace the BP operator on $t, starting from the initial point p [0] = ^ • 1. Let 
p [i] = BP'^(^ [0]) G Af ((f>) be the result of the first £ iterations of BP for ^ > 1. We say that a variable x e Vt+i 
is i-biased if there is a clause a € N{x) such that \px^a{l) [(] ~ ^\ > O.lSt. Let B [£] signify the set of all 
£-biased variables. The core of the proof is to establish the following proposition. 

Proposition 5.1 For all i> 1 we have \B [£]\ < St{n — t). 

Since the values pxA^t,^) are defined in terms of the 'messages' px^a{C,) VA (cf. ©), Proposition 15. 1 1 (with 
£ = uj) and a straight computation yield that ($, id, 1) is (Ji, t)-balanced, as claimed by Theorem l4.3l 



The proof of Proposition IS . 1 l is by induction on (.. Roughly speaking, we need to verify that for 'most' variables 
X all messages ^x~>a{^) [^ + 1] are close to i, assuming that \B [£]| < 5t{n — t). This is the same as saying that 
for most X the ratios 



TT t^^iymti (10) 



areclose to one, because by construction /i2;^a(l) [^ + l]+/ix^a(0) [^+1] = l,cf. (O. (One also has to consider 
the case that the denominator is zero, but this is a mere technicality.) To analyze the product ( fTOl i. we decompose 
it into two pieces, the first of which, denoted by P<i{x -^ a), comprises the factors for b e iV<i (x, B [^] ) \ {a} 
(cf. (|9]l). The following lemma summarizes the analysis of P<i(a; -^ a). (The analysis of the product P>i(x -^ a) 
encompassing the remaining factors is omitted.) 

Lemma 5.2 For all but 0.l5t{n — t) variables x S Vj+i we /zave maX(,g^(^-) |lnP<i(a; — )• a)\ < O.QlSt- 

To prove Lemma I7!2l it is convenient to define A^^a ~ fJ-x^ai^) [^] — 5 G [^1/2, 1/2] . Moreover, we say 
that X G Vt+i is i-good if it has the following three properties (cf. Q0-Q2 in Figurellli. 

GO. X does not occur in a redundant clause. 

Gl. {ekf6t J2beN{x) 2-l^('')l < 1, and 0.16ifc < \N{b)\ < lOOk for all b e N{x). 

For £-good variables x we are going to approximate lnP<i(x — > a) by the linear expression 



&eAf<i(x,BM)\{a} 



sign {x, b)+ E sign (a;, b) sign (y, 6) Ay^b 

yeN{b)\{x} 



(11) 



which is nothing but the derivative of — In P<i{x -^ a). A straight (if lengthy) computation yields 

Lemma 5.3 For any £- good x G Vt+i and any a G N{x) we have \Lx^a + lnP<i(a; — > a)| < 10~^St. 

Proof of Lemma 13721 (sketch ). We can confine ourselves to ^-good variables, because conditions QO, Ql, and 
Q2 ensure the all but 0.05St{n — t) variables are £-good. Due to Lemma |531 for £-good variables it suffices to 
study the linear expression (fTTT i. For any ^-good x the term X^bejv (x B\i])\{a\ 2^^'^''''''sign (a;, 6) is bounded 
by 0.002(5t in absolute value by G2. Since this is the first bit of L^^a, we are left to bound 

^x^a= Y. E 2i-I^WIsign(x,6)sign(y,fe)A,^6. (12) 

b£N<:i{x,Bli])\{a} y£N(b)\{x} 

for £-good X G 14+1 and a G N{x). 

The analysis of the ^x^a is via the linear operator A^f^j from condition Q4, whose definition clearly bears 
some resemblance to the r.h.s. of (fT2l i. The issue is that the dimension of the argument vector A of (fT2l i is 
J2xev 1^(^)1 (it has one component for each edge of the factor graph), whereas A^j^j only 'lives' in dimension 
I Vt+i I = n — t. Thus, to invoke Q4 we will first have to project the vector A down to the lower dimensional space 
R^*+i, and then show that this projection is not too distorting. 

For any y G Vt+i let Af{y) = \JxeVt+i\{y} ^(v) ^ ^<ii^^ B [€]) be the set of all clauses b G N{y) such 
that the entry Ay^b contributes to some term ^x^a- Then, for any y G Vt+i choose by G Af{y) arbitrarily and 
set Dy = Aj,_j.f, [i] (and if Af{y) = 0, then set Dy = 0). The resulting 'projected' vector D = {Dy)y^Vt+i has 
norm Hi^llo^ < ^, because \Ay^i,y [^] | < 5 by definition. An elementary but lengthy argument shows that 

\Dy - Ay^b\ < S^ for any b G AA(y). (13) 

We can easily use Q4 to analyze AB[e]D: let 

C = iCx)xGV,+, = Abii]D, so that Cx= Y. J2 2i-l^('')lsign(a;, 6)sign(y, b)Dy. 

beN<i{x,Ble]) yeN{b)\{x} 



As Q4 ensures that ||Ab[£]|||^ < Sf{n-t), we obtain ||C|li = ||Ab[£]-D||^ < ||Ab[£]||q \\D\\^_ < Sf{n-t). Since 
IICIIi =Exey,+i IG|, we see that 

\{xeVt+i:\C.\>5f}\ <SUn~t). (14) 

To complete the proof, we need to relate C^x and £,x^a for a G N{x) for £-good x. Just from the definitions of 
Ca: and £,x^a we see that 

VeN{a) beN^i{x,B[e])\{a} yeN{b)\{x} 

? |7V(a)|2-l^('^)l+<53 E |Af(6)|2i-l^('')l. (15) 

heJV<i(a-,B[£])\{a} 

Since \N{a)\ > O.Wk for all a e N{x) by Gl, we get |7V(a)|2-l^(°)l < exp(-2c6'fc) = (^^^ < (^t/lO^, 
provided the constant c > in the definition (|6]l of St is chosen sufficiently small and 9k > kr/2'^ > po for a 
sufficiently large constant po > 0. Similarly, Gl yields 6^ Efcew<i(^,B[£])\{a} \Nib)\2^-^^^'>'>^ < 6t/10^. Hence, 
l^x^a — Cx\ < ^t/5000 for any £-good x. Thus, the assertion follows from (fT4l i. D 
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Appendix 

Appendix lAl contains a somewhat broader discussion of Belief and Survey Propagation and their statistical me- 
chanics context, which has been omitted from the main body of the paper for lack of space. 

The other appendices contain the complete proofs, as required by the submission guidelines. Appendix IBJ 
deals with the proof of Proposition l3.2l and AppendixICJcontains the proof of Corollarv l3.4l Then, in AppendixlD] 
we show how Theorem I 1 . ll follows from Proposition l3.2l and Corollary 13. 41 The proof of Proposition l4.2l is given 
in Appendix|El Further, Appendix|F]gives a self-contained proof of Theorem l4.3l (it can be read independently of 
the outline given in Section|5]l. Finally, the proof of CoroUarv Il .6l is given in Appendix iGl 

A Message passing algorithms and statistical mechanics 
A.l Replica symmetry breaking and all that 



Following the non-rigorous paper ||20| , we discuss in this section the statistical mechanics motivation for Hy- 
pothesis Fl.3l which motivates BPdec. Let $ be a fc-CNF with variables V ~ {xi, . . . , x„}. From a statistical 
mechanics perspective, the natural object of study is the set S{^) of all satisfying assignments of $, or, more 
precisely, the uniform probability distribution on this set. The uniform distribution on S{^) is called the Gibbs 
measure g^jj For a random fc-CNF $ the Gibbs measure 5$ becomes a random object, and the aim is to analyze 
the properties that it exhibits with probability 1 — o(l) over the choice of $. 

As the clause/variable density r increases, the Gibbs measure g^ undergoes several phase transitions. The first 
of these is called the Gibbs uniqueness transition. This is the point where the set S'(#) of all satisftying assign- 
ments forms a connected subgraph of the Hamming cube {0, 1} w.h.p. The paper ifZSl establishes rigorously that 
this transition occurs at r ~^ ?■„ = ^T^- ^^^^ result implies the two statements of Hypothesis 1 1 .31 hold true for 
r < ru with t = 1, i.e., for the undecimated formula $. In fact, this is the maximum value of r for which this is 
(rigorously) known to be the case. 

For densities r > ^Jiii but below the fc-SAT threshold rs ^ 2'^ln2 the subgraph of the Hamming cube 
induced on S{^) is expected to be disconnected. But for r < r^ = 2^^ ln(fc)/fc the set S{^) is conjectured to be 
dominated by a single connected component w.h.p. That is, there is a subset C C S{^) that induces a connected 
subgraph of the Hamming cube such that \C\ = (1 — o(l))|5'(#)| w.h.p. In fact, Markov chains such as 'Glauber 
dynamics' are expected to mix rapidly on the dominant component C, and the two statements in Hypothesis [T3] 
are expected to be true for t = 1. However, none of these statements has been proven rigorously. 

If in fact the set S{^) is so 'uniform' that simple Markov chains mix rapidly on C, then it is plausible to expect 
that simple local search algorithms such as simulated annealing might find a satisfying assignment in polynomial 
time. Indeed, a specifically tailored local search algorithm is rigorously known to succeed up to r ~ r^ w.h.p. IIT2I. 
This is, for general k, the largest density for which a polynomial time algorithm is known to find a satisfying 
assignment w.h.p. 

For rd < r < rs — 0{1) a phenomenon called dynamic replica symmetry breaking ('dRSB') complicates 
matters. This phenomenon was (to an extent) established rigorously in JSJ- The set S{^) does not have a domi- 
nant component anymore, but shatters into exponentially many, exponentially small components whose pairwise 
Hamming distance is as large as J7(n) w.h.p. Moreover, each of the small components is 'rigid' in the sense that 
almost all variables take the same value in all satisfying assignments in that component. If we let Ci , . . . , Cn 
denote the components of 5'(^), then the Gibbs measure g^ can be decomposed as 

N 



5* = X! '^' ■ 3c. 



i=l 

where Ci ~ |Ci|/|S'($)| is the relative size of component Ci, and gc^ is the unifom distribution over C;. (That is, 
gCi assigns probabiliy l/|Ci| to each a £ Ci.) In other words, one can think of g^ as consisting of two 'layers': 
firstly, there is a distribution over the components Ci, . . . , Cn induced by the coefficients Ci, and secondly there 
is the uniform distribution g^ inside of each component. Of course, if the sizes \Ci\ of the components were 
concentrated (on a logarithmic scale), then the Gibbs measure would be 'close' (say, within o(l) in total variation 



^^ Actually the Gibbs measure comes with a temperature parameter, which we have omitted for the sake of simlicity. The uniform distribution 
on S{^) is the Gibbs measure at zero temperature. 



distance) to the measure 

1 ^ 

without the leading coefficients c; w.h.p. The two statements of Hypothesis [T3] are expected to be true for t = 1 
for densities r such that the two measures g^ and u# are close w.h.p. 

The paper 1201 claims a further transition called condensation or one-step replica symmetry breaking at r ~ 
Tc = 2*^ In 2 — I In 2 where the measures g^, and U4> cease to be close w.h.p. That is, for Vc < r < Vs the 
coefficients c; are no longer concentrated, but there is a subset V C [N] of size V = cxp(o(n)) such that 
SiGD Ci = ^ — o(l) w.h.p. For densities re < r < Vg Hypothesis II. 31 is not expected to hold, even for t ~ I. 
However, the existence of the condensation transition has not been established rigorously. In fact, Tc exceeds the 
density 2*^ In 2 — A: for which $ is known to be satisfiable Q. According to 1201 the threshold for the existence of 
satisfying assignments is r., = 2'^ ln2 — ^(1 + In 2). 

This picture suggests that BPdec will not succeed to find satisfying assignments for densities Vc < r < 
fs I20II . However, Theorem 11.11 shows that BPdec fails w.h.p. for densities r = 0{2^ /k) even below r^ ^ 
2^ ln(fc)/fc. As pointed out (non-rigorously) in l27ll . the reason for this is that the above picture neglects the effect 
of the decimation process. Recall that in each iteration BPdec decimates the formula by assigning a truth value 
to one variable (for good) and simplifying. Hence, as BPdec proceeds the lenghts of the clauses decrease. In 
other words, decimation imposes stronger contraints on the remaining, unassigned variables. As a consequence, 
the decimation process has a similar effect as increasing the clause/variable density and indeed, according to l27ll . 
decimation brings the condensation phenomenon about even for densities 7' = 0{2''/k) < Vc- (As we will show in 
the full version of this work, the techniques of the present paper can be used to establish this assertion rigorously.) 
Thus, even though for densities r < Tc Hvpothesis ll.3l mav be true for f = 1, it ceases to hold for t>{\~ r/2'°)n. 

Finally, let us comment on Survey Propagation guided decimation, another message passing algorithm for 
finding satisfying assignments of $ l8l l2TII . Recall that BPdec is an attempt at implementing the idealized Exper- 
iment [TT2l In the present notation. Experiment ! 1.21 samples a satisfying from the Gibbs measure gq,. Clearly, one 
could think of a similar experiment that yields a sample from the measure u^ by 'just' replacing the marginals 
Mxt ($t_i) of g#j_i by the corresponding marginals Uxt (4>t_i) of uq,^_^. Survey Propagation guided decima- 
tion is basically an attempt at implementing this latter experiment by means of a message passing algorithm. 

The motivation behind Survey Propagation is that the statement about the marginals Ux^ (^t^i ) corresponding 
to Hypothesis 11.31 might be true for a larger range of r (and t). For, as shown in l20ll27l . the basic obstacle to 
Hypothesis 1 1 . 31 being true is the condensation phenomenon, i.e., the fact that the measure 5$ is dominated by a 
sub-exponential number of components Ci. But by construction this cannot happen in the case of u#, unless, of 
course, the total number A^ of components is sub-exponential. 

The above discussion describes the asymptotic picture for general clause lenghts k. Its qualitiative conclusions 
can be verified to an extent already for k > 10 IS). However, generally, random fc-SAT for 'small' k (say, k < 6) 
appears to behave somewhat differently (see the experimental study |7|). Technically an analysis for, say, k < 6 
is quite challenging, not least because the relative error in the estimate of the fc-SAT threshold obtained through 
the second moment method is larger than for general k [5]. 

A.2 Variations on BPdec 

There are various different ways of implementing the basic idea of combining Belief Propagation with a decima- 
tion procedure. The algorithm BPdec from Section[T]is the simplest but also arguably the most natural variant. 
Other variants used in experiments lT9ll20ll27l differ in 

a. the number uj of iterations of the BP operator used in Step 2, and/or 

b. the order in which the variables are assigned. 

While for the sake of simplicity BPdec assumes that the number of iterations is given by some fixed function 
Lj = uj{n), it could make sense to use a 'dynamic' criterion such as the difference between the message vectors 
fi [i] and /i [^ — 1] in some norm. For instance, one could run BP until it reaches an (approximate) fixed point. 
The present analysis extends easily to this variant. Amending the proof accordingly is straightforward, because 
the key statement Proposition 15 . II on the dyanmics of BP holds for any £. That said, according to Hypothesis [Tj] 
(which inspired the use of Belief Propagation in the first place), the number cj of iterations should not matter, so 
long as 1 <C cj <C In n. 



With respect to b., sometimes the variable x for which |i — fix{^t-i,^)\ is largest is chosen to be assigned 
at iteration t. The reason given for this choice is essentially numerical stability: numerical errors might disperse 
'hard' constraints such as unit clauses. However, conceptually the choice of the variable to be assigned next should 
be unimportant, because the (ideal) BP operator does respect such hard constraints implicitly. 

An interesting open problem is to generalize the present analysis to Survey Propagation. Furthermore, it would 
be interesting to see if the techniques developed here to prove a positive result about BPdec. 



B Proof of Proposition 1X2 



We consider an additional variant of BPdec that receives the order tt in which variables are to be decimated as an 
input parameter. 

Algorithm B.l BPdec($,7r) 

Input: A fc-SAT formula $ on 1/ = {xi , . . . , a;„}, a permutation n G Sn- 
Output: An assignment t : V ^ {0, 1}. 

1 . Let $0 = $. 

1. Fori = 1, ... ,n do 

2. Compute the BP results fia:{^t^i,Lo)- 

3. Let CT(a::^(t)) = 1 with probability ^a:($t-i,t»j), and se{a{x^(t)) = otherwise. 

4. Obtain <l>t from <I>t_i by substituting the value (7{x^^t)) for a;^(i). 

5. Return the assignment a. 

Fix a fc-CNF $ that is {6, t, ^)-uniform for all 1 < t < T. Let 5„ be the set of all permutations on [n]. Let 
A be the probability distribution on pairs (tt, cr) g 5„ x {0,1} induced by choosing a permutation tt e 5'„ 
uniformly and letting cr = BPdec(<i>, tt). Then /3$ is the cr-marginal of A, whence 

i3^{£) = X{Sn X £) for any £ C {0, 1}^ . (16) 

In order to study A, we consider another distribution A' on pairs (tTjCt') g Sn x {0, 1} that is easier to 
analyze and that will turn out to be 'close' to A. To define A', let Bt be the set of all pairs (tt, a) such that ($, tt, a) 
is not ((5f , t)-balanced. Moreover, let B ~ Ut=o ^t- ^^ distribution A' is induced by choosing a permutation tt 
uniformly at random and running the following algorithm on $, tt. 

Algorithm B.2 BPdec'($,7r) 

Input: A fc-SAT formula $ on V^ = {xi , . . . , a;„}, a permutation tt G Sn. 
Output: An assignment cr' : V ^ {0, 1}. 

0. Let $0 = $. 

1. Fori = 1, ... ,n do 

2. Compute the BP results Hx{^t-i,io). 

3. If ($, TV, a') is {5t, t)-balanced, then 

let CT'(a;^(t)) = 1 with probability ^i.^(jj($t-i,tj)), and set o-'(a;^(t)) = otherwise, 
else 

let er'{x^(^t)) = C with probability ^ for ( = 0, 1. 

4. Obtain $* from $t-i by substituting the value o-'(a;^(i)) for a;^(i) and simplifying. 

5. Output the assignment o-'. 

Roughly speaking, BPdec' disregards the outcome p of BP if it strays too far from the 'flat' vector ^1. We 
claim that A and A' are related as follows. For T C SnX {0,1} we let J^t be the set of all pairs (tt, a) for which 
there is (7r*,cr*) G J- such that 7r*(t) = 7r(t) and fT*(a;^(t)) — a{x.^i^t")) for all 1 < t < T. Intuitively, J^t is the 
projection of T on the first T steps. We are going to show that 

X{T) < A'(J't) + A'(6) forany J-C Sn X {0,1}^. (17) 



To establish (fTTb . note that by construction for any (tt, a) ^ Bt and any C, G {0, 1} we have 

A [cr(a;^(t+i)) = Ck = tt A Vs < t : (t(x^(,,)) = a{x^(s))\ 

= A' [(T'(.T^(t+i)) = Ck = TT A Vs < t : (T'(a::^(s)) = cr(x^(s))] . 

Consequently, for any pair (tt, cr) ^ S we get 

A [Vt < T : 7r(t) = 7r(t) A (T(t) = cr(f)] = A' [Vf < T : 7r(t) = 7r(t) A cr'(t) = cr(t)] . (18) 

In particular, X{B) ~ A'(,B). Hence, for any event F we obtain 

A(J-) < A(J-t)< A(J-T\6) + A(B)'i'A'(J-T\B) + A'(6)< A'(J-T) + A'(e), 

thereby proving (fTTb . 

Let A" be the uniform probability distribution on Sn x {0, 1} , and let (tt, u) be a uniformly random pair. 
We are going to relate A' and A". To this end, let At be equal to one if (tt, ct) ^ Bt and a;7r(t) is {5t, f)-biased in 
(<1>, TT, a). In addition, let A = ^t<T ^t- ^^ claim that for any pair (tt, a) 

A' [Vt < T : 7r(t) = 7r(f) A <T'(a;,(t)) = (7(a;,(t))] 

< A"[Vi<T:7r(f) = ^(t)AM(x,(t)) = a(x,(t))] •2^("'")[];i + 2,5t. (19) 

To see this, fix any pair (tt, cr) and let Ct = {(7r*,(T*) : TT*{t) = 7r(t) A o-*(a;7r(t)) = cr(x^(t))}. Let 1 < t < T. 
We can bound the conditional probability A' [£tk(^) = ""(i) ^ A5<t ^s] ^^ follows. 

Case 1: {tt,<t) G Bt- In this case ((E'jTrjCr) is not (5t,t)-balanced. Therefore, step3ofBPdec' chooses the value 
(T'{xT,-(t)) uniformly. Hence, the event <T'{x.^(t)) ~ cr(2^ir(t)) occurs with probability i. 

Case 2: (tTjCt) ^ Bt and At{n, a) =0. Since ($,7r, ct) is ((5t,i)-balanced, step3of BPdec' uses the BP marginals 
fj,x^,t) (C) in order to assign x^(t). Because At{TT, ct) = 0, the variable X7r(t) is not {6t,t)-hiased, whence 
Ma;,r(t)(C) ^ 5 + (5t for C = 0, 1. Hence, the probability that (t'(x^(()) = CT(a;7r(t)) is bounded by ^ + Jj. 

Case 3: At{TT,a) = 1. In this case we just use the trivial fact that the probability of the event cr'(xjr(t)) = 

ct(x^(j)) is bounded by 1 < 2(i + St). 

In any case, we obtain the bound A' [Ct\TT{t) = 7r(i) A As<t ^s] < 2'^''^'^''^\^ + 5t). Consequently, as A" is the 
uniform distribution, we get 

X'[Ct\n{t)^n{t)AA^^,L,] ^ ^^^(,,,) 
A"[AkW = 'r(i)AA,<,£.] -^ ^ ^*^- 

Multiplying this estimate up for t <T,we obtain ( fT9b . 

To put ( fT9b to work, we need to estimate the random variable A{tt, cr'). Let Ej- = J2s<t ^s- We claim that 

A' [Ai-K, a') > 4(St + ^n)] < e^-^n). (20) 

To see this, we are going to bound the probability that At{TT,(T') = 1 given the values n{s), (T'{xTr(^s)) for 
1 < s < t. 

Case 1: the event Bt occurs. Then At = by definition. 

Case 2: the event Bt does not occur. In this case ($, tt, ct) is ((5*, t)-balanced, which means that no more than 
St (n — t) variables are biased. Since the permutation tt is chosen uniformly at random, the probability that 
a;7r(t) is (5i,t)-biased is bounded by (5i. 

Thus, in either case the conditional probability of the event At = lis bounded by St. This implies that the random 
variable A{tv, cr') — J2t<T ^ti^^, cr') is stochastically dominated by a sum of mutually independent Bernoulli 
variables with means 6i, . . . , 6t- Therefore, (l20l i follows from Chernoff bounds. 



Finally, combining ( fT9] l and ( l20l i. we see that 

A' [Ft] < A' [At > 4(Et + C")] + A' [J"t A At < 4(Et + Cf^)] 
< exp(-en) + A" [J-t] • 24(^^+«") [] 1 + 2^* 



t<T 



< A" [Tt] ■ cxp(6S]t + 4^?i) + exp(-^n) for any J" C S'„ x {0, 1}* 



(21) 



Our assumption that <1> is {S, t, ^)-uniform ensures that X"{Bt) < exp(— 10(^7i + S^)) for all t <T. Together 
with (l2n i. this implies that 

A' [Bt] < A" [Bt] cxp(6St + ^£,n) + cxp(-^n) < 2 cxp(-^n) for any t < T. 

Therefore, 

A' [B] < 2Texp(-^n) < exp(-0.9Cn). (22) 

Putting things together, we obtain for any £ C {0, 1} 

P^{£) = X[SnX£] 

< A' [(S*,! X £)t] + A' [6] [due to ^] 

< A' [{Sn X £)t] + cxp(-0.9e") [by (123] 

< A" [(5„ X £)t] exp(6(Sr + ^n)) + exp(-0.9$7i) [by (EB] 

|(5„x£:)t| 



il2^' 



exp(6(I]T + ^JT-)) + exp(— 0.9^n) [as A" is uniform] 



\£\ 



< -^ • cxp(6(St + (,n)) + cxp(— 0.9^n) [by the definition of £t], 



as desired. 



C Proof of Corollary |M 



For 1 <t <T and a k-CNF $ we let Xt($) signify the number of pairs (tt, a) e Sn x {0, 1} such that ($, tt, a) 
is not ((5t, t)-balanced. Then Xf ($) = 2"n! • P [($, tt, u) is ((5t, t)-balanced] , whence 

$ is (5,t,C)-uniformiff Xf($) < 2"n! • cxp(-^n - 4At). (23) 

With respect to the random formula $, Theorem [33] yields E [X4($)] < 2"n! • exp(— 3^ — 4At), for a certain 
^ > 0. Hence, Markov's inequality and the union bound entail 

P [3t < T : Xt{^) > exp(-^ - 4At)] < ncxp(-2^ri) < cxp(-^n), 

and thus the assertion follows from (l23l l. 



D Proof of Theorem O 

Let us keep the symbols introduced in Theorem l3.3l Let U be the event that # is {d, t, ^)-uniform for all 1 < t < 
T. Let 5 be the event that |5(#)| < n- E|5($)|, i.e., the number |S'(*)| of satisfying assignments of* does not 
exceed its expectation too much. By Corollary [33] and Markov's inequality, we have ^ E UOS w.h.p. Given that 
U and S occur, Proposition l3.2l implies 



^*(5'(*)) < n-E|S'(*)|-2-' exp 



4E^* 



+ cxp(-^n/2). 



(24) 



Simple calculations show J^Ui ^t -- ri/(cfccxp(cfcr/2'=)) and E|S'(*)| = 2"(1 - 2^'=)'" < 2"cxp(-rn/2'=) 
Plugging these estimates into (i24l i and recalling that T — {1 ~ r/2^)n, we get 

r exp(— cfcr/2'') 



/3*(5(*)) < exp 
< cxp 



(l+o{l))n\\u2-- + 
(l + o(l))n(ln2-l 



ck 
cxp(— c/cr/2'^) 
ck 



-T +cxp(-^7i/2) 

T +cxp(-^n/2) < cxp(-0(n)), 



provided that kr/2^ > po for a large enough constant po > 0. Finally, Theorem I 1 . 1 [ follows from Fact l3.1 



E Proof of Proposition 14^ 



As in Proposition 14.21 we let Sg = cxp(— c(l — s/n)k) for a small enough c > and any 1 < s < n, and we 
assume that p = kr/2^ > po for some large enough number po satisfying po > exp(l/c). Let T = (1 — p/k)n 
and suppose that 1 < i < T. Set 9 = 1 — t/n and observe that 9k > p. Let 6 = St. Let #4 be the formula obtain 
from $ by substituting the value 'true' for xi, . . . , Xt-i and simplifying. Since the 5s form a geometric sequence, 
we have 

-^ '^ ck exp(c0fc) ' 

Observe that 

95n > lO^^Et 

if P > Po is chosen sufficiently large. 

Let fci ~ y/c6k (with c > as in ^). Moreover, for a variable x E Vt and a set Q C Vt let 

Ni{x,Q) = {beN{x):\N{b)\Q\>O.Ol{l-t/n)k,\N{b)r]Q\x\ = l}, 
Nyi{x,Q) = {be N{x) ■.\Nib)\Q\>O.Ol{l-t/n)k,\N{b)nQ\x\>l} 

To prove Proposition 14.21 we will study two slightly different models of random fc-CNFs. In the first model 
$', we obtain a k-CNF by including each of the (2?i)'^ possible clauses over V = {xi, . . . , x„} with probability 
p = m/{2n)'^ independently. Thus, $' is a random set of clauses, and E|$'| = m. In the second model, we 
choose a sequence $" of m independent fc-clauses $"(1), . . . , ^"{m). Thus, the probability of each individual 
sequence is (2n)^'°'". The sequence $" corresponds to the fc-CNF {$"(1), . . . , ^"{m)} with almost m clauses. 
(Recall that we defined a fc-CNF as a set of clauses.) The following well-known fact relates $ to $', $". 

Fact E.l Let £ be any property of random k-CNFs. Then 

P [* e £] < C^M •?[*'€£■], 
?[*€£] < (1 + 0(1)) •?[*"€£]. 

Proof. The total number of clauses in $' is binomially distributed with mean m. Stirling's formula implies that 
the probability that $' has precisely m clauses is at least rt{l/y/m). Furthermore, the expected number of pairs 
I <i < j <m such that *"(i) = *"(j) is o(l). Therefore, P [| {*"(1), . . . , *"(to)} \ = m] = 1 - o(l), and 
given that this occurs, the set {$"(1), . . . , ^"{m)} has the same distribution as $. D 

Due to Fact lE.ll it suffices to prove that the statements Q0-Q4 hold in either models $', $" with probability 
at least 1 — exp(— IIEt). 

E.l Proof of QO. 

A direct computation shows that the expected number of redundant clauses is 0(1). Furthermore, modifying a 
single clause of #" can change the total number of redundant clauses by at most two, i.e., the number of redundant 
clauses is 2-Lipschitz. Therefore, Azuma's inequality implies the desired bound. With respect to the degrees of 
the variables, simply note that the total number of clauses m equals the sum of the degrees of the variables divided 
by fc. 

E.2 Proof of Ql. 

Working with the model #', we are going to prove the bound on the number of variables in very short/very long 
clauses first. For any j the expected number of clauses of length j is 

A,.2^n.^(i-«)'-..|i^?i^(*':;)«.-'(i-«r^ 



Let's begin with short clauses of lengths j < 9k /lO. The total expected number of such clauses is bounded by 



-)0.iefc. 



P[Biii(fc-l,6') <ek/10] 



j<ek/io 

< 2"-^^''p9n ■ cxp(-0fc/3) < 6'cxp(-6'fc/4)n, 



provided that 9k > pis sufficiently large. Hence, by Chernoff bounds the probability that in total there are more 
than 9n ■ lQ^^S/{9k) clauses of length < 0.19k is bounded by 



exp 



10-5(5 
29k 



In 



10-^5/{9k) 
'cxp(-6'fc/4)n 



9n 



< cxp 



3 • 10^9k 



9k-9n\ < cxp(-10-°6lfc 



Moreover, if there are fewer than 9n ■ li)^^ 5 / {9k) short clauses, then these do not contain more than 9n ■ IQ^^S 
variables. 

To bound the number of variables in clauses of length > lO^fc, we use the following argument. For a given 
/^ > let C^ be the event that $f has p. clauses so that the sum of the lengths of these clauses is at least A = 109kp. 
Then 



P [^p] < 



m\fkp\. (I 



A 



+ ( 



k^ — X 



(Explanation: choose the p clauses; out of the kp literals that they contain, choose A whose underlying variables 
have indices > (1 — 9)n; since none of the p has been eliminated from $t, the remaining kp — X literals must 
either be negative or their underlying variables are in Vt.) Thus, 



P[^m] < 



:)Cr)<->^(i 



A;/.i 



< 






) 


A2 


- + 9 


)(l) 


1091 * 


M / 

< 

v 


cnp\^ 
kp) 


/C\106»" 

(l + 2.)(-) 


< 


fenp\'^ 
\kp J 


exp{29) (-) 


-ifcA' 

< 


\ kp J V 4 / 


< 




enp /e\io®'= 
kp UJ 


A* 




= 




'enp- I09k /e\ioefc- 
kX UJ 


1^ 


A UJ 


\/{109k) 


= 




_[ X J U)_ 


A 




, for A > 10-^9Sn we 


get 




P[^,] 


- \pj\xj' \2 J 






< 


" / mnn \ l/lOSfe 

/'lOOOep^ ' fe\ 
\ S UJ 


-| A 
< 


[iy<eM-^0-'°OSn), 








\ 




^ 






_ 









kfi 



provided that p> pa is sufficiently large. In summary, we have shown that 



xeVf.Bbe Nix) -.^-^^^[0.1,10] 



> lO-^9Sn 



< cxp(-10-"6'5n) 

< cxp(-100I]t). 



Finally, we need to bound the number of variables x E Vt for which 5{9k)^ J^beNix' 



2-l^(b)l > 1. Let 



(25) 



(26) 



be the expected number of clauses of length j that contain a given variable x E Vt in the formula $t. Let Xj (x) 
be the actual number, and let Xji (x) be the number of such clauses where x appears in position I in that clause 
(1 < I < j)- Then by Chernoff bounds 

p [Xji{x) > loifi-j + 5-\0ky^)] < cxp{~io/{5{ek)^)). 

Let ( = ex.p{-l0/{6{6k)^)). For j, I let Vji be the set of all variables x e Vt such that Xji{x) > 10(^j + 
d~^{9k)~'^). Since the random variables {Xji{x))x£Vt are mutually independent, Chernoff bounds yield 



(27) 



provided that p > pois sufficiently large. Furthermore, if a; ^ Vji for any 1 < j < lQ9k and any 1 < ^ < j, then 
for p > po sufficiently large 

^ 2-i^wi < 10 ^ 2-^ip, + s-\ek)-^) 

bGN(x):\N{b)\<10ek j<10ek 

< ios~\eky^ + 10 Y^ 2-J/ij 

jKiaek 

< 10S-\dk)-^ + lOp < 20S-\9k)-'^ < s-\0ky^. 

Hence, the assertion follows from (|27] | and our bound on the number of variables in clauses of length > 109k. 



\V,i 


9Sn ' 
^ {9kf 


< cxp 

< exp 


9dn , 


/ 96n 


\] 




2(0fc)9 

" 9Sn ■ 


< exp 


)\ 


9n 






{9kY^ 



E.3 Proof of Q2. 

We will work with the model $' once more. Let Q C Vi be a set of size \Q\ < 95n. We first deal with the 
sums over b S A^>i(x, Q) for x E Vt- Given our bound on the total number of variables in clauses of lengths 
smaller than 0.\9k or greater than 109k, we may assume that x occurs in no such clause. For a variable x we let 
Q{x, i,j, I) be the number of clauses b such that clause b contains x (or x) in position i in the decimated formula, 
0.19k < \N{b)\ == j < 109k, and \N{b) n Q \ a;| = I. Then Q{x, i,j, I) is binomial and for / > 2 we have for 
p> Po sufficiently large 

mix, I, J, I) < (^j)<5V, < p(^)'5'2^ < 2=-'6'\ 

with /ij as defined in ( l26l l. Hence, by Chernoff for j — I > kiwe get 

P[Qix,i,j,l) > 10-2^-^S^-^] < exp(-2-'-'(5i-9) < cxp{-2''^S^-^) < exp{- exp{c^/^9k)). 

Let Z{i,j, I) be the number of variables x for which Q{x, i,j, I) > 10 ■ 2^~'(5^-^. Then Z{i,j, I) is binomial 
(because we fix the position i where x occurs) and EZ{i,j, I) < 9ncxp{~ cxp{c^/^9k)). Hence, by Chernoff 
bounds 



P[Z{i,j,l)>9Sn/{9k)^] < cxp 

< cxp 



95n 



m- 



■In 



95n/{9kY 



96n 



9n exp(— exp{c'^/^9k)) 



m 



- ■ cxp{c^^'^9k) 



(28) 



Furthermore, if x is a variable such that Q{x,i,j,l) < 10 • 2-' '(5^'^ for a\\l<i<j,2<l<j — fci, and 
O.l^fc < j < 109k, then for p > pa sufficiently large 



y^ 2\N{b)nQ\x\-\N{b)\ 

b£N^i{x,Q) 



0.iek<j<lQek 2<l<j-ki 



Finally, by the union bound and 
i,j,l is bounded by 



esn 



exp 



the probability that there is a set Q such that Z{i,j, I) > 05n/{9k)'^ for any 



{OkY 



■ cxpic^'-'ek) 



< cxp 



< exp 



o{n) + e5n{l - \n{e5)) 



esr 



{OkY 

o(l)-21n(5-exp(c3/46'fc) 



■ cxp(c2/36'fc) 



< exp[-100Et], 

provided that p > po is sufficiently large. This implies the claim regarding the middle sum in Q2. 

The second part of the proof is about clauses b £ N{x) such that \N{h) n Q \ .t| = 1. In this case a similar 
argument applies, but we need to work with ? = 1. We have 

^Q{x,^,]^)<j5^lJ <pj62^. 

For O.iek <j< lOek Chernoff bounds yield 

P [Q{x,iJ,l) > 10pjS2^] < cxp(-exp(c2/36»fc)). 

Let Z{i,j) be the number of variables x £ Vt such that Q{x,i,j, 1) > 10pj62^. Then Z{i,j) is binomial with 
EZ{i,j) < 6nexp{— exp(c^/'^0fc)), and thus Chernoff bounds yield 

P [Z{i,j) > eSn/{ek)'^] < cxp [-95n] . 

Hence, taking the union bound over Q, i, j, we see that with probability > 1 — cxp(— lOOSt) there are no 
more than 65n/{9k) variables x such that Q(a;, i,j, 1) > 10pj(52-' for any i,j. Moreover, if x is such that 
Q{x,i,j,l) > 10pjS2^ for all i, j, then 



beNi(x,Q) 



< 



Y^ WfpS < {0kfS, 

0.iek<j<10ak 



as desired. 

We come to the third sum in Q2. Once more, we fix a set Q of size \Q\ < 05n. As before, we may disregard 
variables that occur in clauses of lengths smaller than Q.19k or greater than lQ9k. Thus, fix a clause length 
Q.\9k < j < 109k and a position 1 < I < j- For a variable x £ Vt let A/'+(x, j, /) be the number of clauses 
b G N{x) such that |A^(fc)| = j, Xi occurs in position I in b positively, and \N{b) n Q \ a;| < 1. Define 
Af-{x,j,l) similarly for negative occurrences of x. Then E(A/'+ + A/"-) < Pj < 2^ p (with pj as in (l26ll). 
Furthermore, JV+ {x, j,l), M- {x, j, I) are binomially distributed with identical means, because in $ each literal is 
positive/negative with probability i. Hence, for j > 0.19k Chernoff bounds yield 



P [\N+{x,j,l) - N-{x,j,l)\ > 2^d{9k) 



-31 



< 



exp 



< exp 



(2J(5(6ifc)-3)2 
' 3{2U {9 k)-3 + Pj) 
2^6^ 



<cxp(-exp(6'fc/100)), 



4(6ifc)V. 

provided that c > is chosen sufficiently small. For different variables x G Vt the random variables A/+ {x, j,l) — 
M^ {x, j, /) are independent (because we fixed the position I where x occurs). Hence, the number i5(j, I) of vari- 
ables for which |A/'+(x,j, /) — A/'-(.T, j, /)| > 2^ (5(0^)""^ is binomially distributed with mean < 6'nexp(— exp(O.O10fc)). 
Consequently, Chernoff bounds give 



P[B{j,l)>96n/{9kf] < exp 



< exp 



2 '^ ' \exp{- exp{9k))9n 



9Sn 
"{9kf 



■ exp(6lA:/101) 



<exp[-e'(5ncxp(6lfc/102)] 



provided that p > po is sufficiently large. By the union bound, the probability that there is a set Q and j, I such 
that6(j,0 > 98n/{9kf is bounded by 

n^ilH I cxp[-6iJncxp(6'fc/102)] < cxp \o{n) + 95n{l -M 95)) - 95ncyiY,{9kl 102)] 
\OonJ 

< exp [o{n) - 295 hi{5)n - 96 cxp{9k/102)n] 

< exp[-100Et]. 



Finally, if a; ^ B{j, I) for any j, I, then 

sign(a:;, b) 



E 



beiv<i(x) 



2l^WI 



provided that p> pois large enough. 



J22-'(.M+ix,j,l)-M-ix,j,l)) 



'j,i 



< ^2-J|AA+(x,j,0-AA_(x,j,0| 

3,1 



E.4 Proof of Q3. 

We carry the proof out in the model $". Let 0.01 < z <1 and let Q be a set of size \Q\ — dqn with 0.01(5 < q < 
100(5. In the proof of Ql we actually showed that 

Y^ \N{b)\ < W-^9Sn < W-^\Q\. 

b:2:^^[0.1.10] 

Therefore, here we only need to consider clauses b of lengths O.l^fc < |6| < 109k. 

For integers S,Z > let £{Q,S,Z) be the event that in $t contains a set Z of Z clauses such that 

EbG2 1^(^)1 = ^' O-l^'^ ^ I^WI ^ lO^'^' ^"'l l^(^) f^Ql > ^1^(^)1 for all fe e Z. We claim that for 
given Q, S, Z 



P[£{Q,S,Z)] < {f)[^s){zS^^''''''' 



(1-0)^^-^9^^. 



For there are m clauses in total, out of which we choose Z; then, out of the kZ literal occurrences of these Z 
clauses we choose S whose underlying variables lies in Vt, an event that occurs with probabihty 6 = \Vt\/n 
independently for each. Furthermore, all kZ — S literals whose variables are in T^ \ Vf must be negative, because 
otherwise the corresponding clauses would have been satisfied by setting xi, . . . , Xt^i to true. Finally, out of the 
S literal occurrences in Vt a total of at least zS has an underlying variable from Q, which occurs with probability 
q = \Q\/{6n) independently. 
Hence, we obtain 



¥[£{Q,S,Z)] < (■p2-^-^ 



21/- . £ 



< 



Z 



-kZ 



2iA- . 



^s fkZ 

S 

, m 
< 



zS 



gS^^_g^kZ~S 
2~^^{CqY'^ 



(29) 



for a certain absolute constant C > 0, because z > 0.01. Since all clause lengths are assumed to be between 
O.l^fc and lO^fc, we obtain 0.1S'/(6'fc) < Z < lOS/{0k). Therefore, 



Z 



^-kz < / em \Z /epn\Z / IQeOk- pn 



\2''z) - \kZ J V 



kS 



ios/{ek} 



imqj 



lOS/iek) 



(30) 



Since q < 100(5 = 100cxp(— c6'A;) and 9k > p, we have 1/q > lOOp for p > po sufficiently large. Hence, (l30l l 
yields 



2-kZ ^ -20S/(9k) 

Plugging dSTT i into (|29] l, we obtain for 9k > p> po large enough and S > 1.01|Q|/z 
P [SiQ, 5, Z)] < q--20S/{ek) . (^^^ys < ^ 



0.9999zS ^ 1.009|i 



(31) 



(32) 



Let £(g, S, Z) denote the event that there is a set Q C Vt of size \Q\ = q9n such that £((5, S^ Z) occurs. Then for 
S > l.Qlq9n/z the union bound and ( |32] | yield 



P[Siq,S,Z)] < 



in 
'9n 



^i.oa9qen < p^p ^^Q^^^ - Ing - 1.009 Ing))] < cxp [^0.008qen\nq] . (33) 



Finally, let £ be the event that there is a set Q C Vt of size OMlOSn < 1009Sn and S > 1.01|Q|/z, Z > such 
that £{q, S, Z) occurs. Then (l33T l yields for p> po large enough 



P[£] < 



exp [0.008 • 0.00016'(5nhi(100(5)] < exp(-lOOEt), 



as desired. 

E.5 Proof of Q4. 

We are going to work with the probability distribution $" (sequence of m independent clauses). Let A4 be the set 
of all indices / € [m] such that the ^th clause of $"(/) is still present in the decimated formula at time t, i.e., ^"{l) 
does not contain any of the variables xi, . . . ,xt positively. For each / e A^ let C{1) be the number of literals in 
#"(/) whose underlying variable is in Vt- We may assume without loss of generality that for any I G M the C{1) 
leftmost literals #"(/, i), 1 < « < C{1), are the ones with an underlying variable from Vt- 

Let Q C Vt. Analyzing the operator Ag directly is a little awkward. Therefore, we will decompose Ag 
into a sum of several operators that are easier to investigate. For any clause length 0.19k < L < 106k, and 
^<'i<j<L, Ig Ai, and any distinct x,y £ Vt we define 

r 1 if C{l) = L and {^" {I, i) = x A *" (?, j) = j/) V (*" (/, i) = S A *" (/, j) = y), 
m,yii,j, l,L) = l -1 if C{1) = L and (*" {I, z) ^ x A *" {l,j) = y) V (*" (/, i) ^ x A *" {I J) = y), 
[ otherwise, 

while we let m^-xiijj, /) = 0. Moreover, for x, y € Vt welet ?7i^j,(i, j, L) — J2ie\m] "'^T'xyihj, l,L). For a variable 
X G Vt we let 7V(.T, Q) be the set of alll e M such that 0.10k < C{1) < 109k and the clause ^"{l) contains at 
most one literal whose underlying variable is in Q \ x. We are going to analyze the 1 1 • 1 1 p -norm of the operators 



A^^iR^'^R^S r^ 



J2 E 2-V,^(j,j,L)r, 

beAr(Q,x)yeN{b)\x 



xGV 



The following well-known fact about the norm \\-\\^ of matrices with diagonal entries equal to zero simplifies 
matters a little. For A C Vf we let 1^ G {0, 1}^' denote the indicator of A. 



FactE.2 We have 






< 24max^^Bcy:Ans=0 



A^'-IaAb 



Lemma E.3 Fix i,j, L, Q. With probability at least 1 — cxp(— ^ri) we have 



A%^ 



< S^0n. 



Proof. Fix two sets A,Bc Vt. For each I G A4 and any x,y £ Vt the two 0/1 random variables 
y^ max{'mxy{i,j,l,L),0}, ^ max{-m^j^(i, j, /, L), 0} 

ix,y)£AxB {x,y}£AxB 

are identically distributed, because the clause $" is chosen uniformly at random. In effect, since the clauses 
{^"{l))i^[„i] ^rc mutually independent, each of the two random variables 

p{A,B) = X! X! '^l&^f(x,Q)^'^eix{m^y{i,j,l),0}, 

leM {x,y)eAxB 

v{A,B) = X! X! liGA/'(2;,Q)max{-m:ra(J,j,0>0} 

l£M {x,y)£AxB 

is a sum of independent Bernoulli variables, and the distributions of p,{A, B) and v{A, B) coincide. As each 
of the clauses ^"{l) is chosen uniformly, for each / G [m] the probability that / G A^ and C{1) = L equals 

(l)e^{i-e)''-^2^-'' 



Therefore, we obtain the bound 

E{piA,B)) = E{,y{A,B))<m(''\9^{l- 



\k — Lr^L 



-k _ 2V'^ 



L 



11- 



\k — Lr)L — k 



L \L-1 ^ ' - L 



(34) 



Since both fJ.{A, B) and v{A, B) are sums of independent Bernoulli variables, ( [34b entails that 

Var(Ai(A, B)) = Var(iy(A, B)) < E{^l{A, B)) ^ E(z.(A, B)) < 1_^. 



Hence, Chernoff bounds yield that for a certain constant c > we have 
P 



|/i(v4, B) - E(Ai(A, B))\ > cJ2Lp/L ■ 0n 



Since this is true for any fixed A, B, and because 



AJ^^1b,1a 



the union bound and ( [35T l imply the assertion. 
To complete the proof of Q4, let 



= P 



< 16 



{A, B) - Y.{v{A, B))\> C'j2^p/L ■ On 



'On 



(35) 



1-L 



{li{A,B)~y{A,B)), 



u 



Ab = 



E EE^r- 

0.iefc<L<106ife i=l j^i 



Thus, both Aq and Ag are Vi x Vj -matrices. For (x, y) G Vi x Vj corresponding the entries of A'q, Kq differ only 
if either x or y occurs in a redundant clause. Consequently, by QO 11 Aq — AqII = o{n). Therefore, Q4 follows 
from Lemma IE3] and the union bound over s, t, L, and Q. 



F Proof of Theorem 4.3 



This appendix contains a complete proof of Theorem l4.3l It is self-contained in the sense that it is independent of 
the outline given in Section |5] Throughout this section, we keep the notation from Section^and the assumptions 
ofTheorem \4.3\ To unclutter the notation, we let 5 ~ St- Moreover, let p be such that r = 2'^p/k. 
Let ki = yjcdk (with c > as in ©). 



El Outline 

We are going to trace the BP operator when applied to the initial point (Ua;_j.a(0) [0] = [i^^a^X) [0] = \ for all 
a; e Ft, a e N(x). Let p. [^] = BP'^(^ [0]) G M($) be the result of the first (. iterations of BR Let 

We say that a; G Vt is i-hiased if there is a clause a £ N{x) such that | A^j^a [£] \ > 0.1(5. Clearly, no variable 
is 0-biased. Let B [£] be the set of all ^-biased variables. To prove Theorem l4.3l the core task will be to bound the 
sizes \B[e]\. 

We are going to construct a sequence of supersets T [i] D B [£] whose sizes are easier to estimate. Actually 
we will construct sets of variables Ti [£], T2 [£] and sets of clauses T3 [£] inductively and let T [i] ~ Ti [£] U 
T2 [£] U N{T'i [(]). For £ = we let Ti [0] = T3 [0] = 0. Moreover, let T2 [0] be the set of all variables x such 
that there is a clause b e N{x) that is either redundant, or \N{b)\ < 0.19k, or |A^(6)| > 100k, or that satisfy 

To define T [i* + 1] inductively for ^ > 0, we need a bit of notation: for x ^ V and a G N{x) we let 



N<i{x-^a)[e+l] = N<i{x,T[£])\a, 



(36) 



n 






P<i {x -> a) [^ + 1] = 

b£N<i{x^a)le+l] 

N^i{x^a)[e+1] = {beN{x)\iaUN<iix,T[£])):pb^,{0)[e]^0}, 



P>i {x ^ a) [£ + 1] = 



n 



fce7V>i(x^a)[C+l] 



^^b^xio)[£Y 



It is direction from their definition (|2|i that the terms /if,-).2-(0) [£] are strictly positive for all b E iV<i(a; -^ a), so 
that P<i (x -^ a)[£ + 1] is well-defined. 

The idea behind these definitions is the following. Assume for a moment that iJ.b^x{0) [£] 7^ for all b E 
N{x). Then in order to show that A^^a [^ + 1] = Ma;-i-a(l) [^ + 1] — ^ is close to zero it suffices to verify that 
the ratio 

fi^^ajl) [i + I] -pr ^fc^^(l) [e] Nr« , 11 p f _^ ^^^, 11 ^^^^ 

M..a(o)[£+i] - ^^ n^^^ ii^::jm = "^^^ ^" ^ «) [^ + 1] • ^>^ (- - «) [^ + 1] (37) 

is close to one, because pix^a{0) [^ + 1] + IJ-x~>a{^) [^ + 1] = 1 by construction. Moreover, dJTl i is close to one if 
both factors on the r.h.s. are. 

The set Ti [£ + 1] contains all variables for which P<i (a; — > a) [£ + 1] fails to be close enough to one: 



Ti[£+l]^ {x eV : max |P<i (a; ^ a) [^ + 1] - 1| > 0.01(5 

[ aeN{x) 

To also deal with the second product P^^^a [^ + 1]' we define additional sets T2 [£ + 1], T3 [i + 1]. To define 
T2 [£ + 1], let us say that a variable x is (£ + \)-harmless if it enjoys the following four properties. 

HI. We have 5{0kf 'EbeN(x) 2"l^<^)l < 1, and 0.16lfc < |A^(6)| < lOOk for all b<EN{ 



X . 



H2. EbeN.ix,m) 2-I^WI < {ekfS and E..^>,(.,tM)) 2l^Wn^M\-l-l^('')l < 6/{ek). 
H3. There is at most one clause b e N{x) such that \N{b)\T[e]\ < ki. 



H4. 



E.e^<,wsign(^,&) •2-1^(^)1 



< 0.01(5. 



Let i/ [£ + 1] signify the set of all {£ + l)-harmless variables. Further, let T2(£ + 1) be the set of all variables x 
that have at least one of the following properties. 

T2a. There is a clause 6 e iV(a:;) that is either redundant, or |iV(6)| < 0.16*^;, or |7V(6)| > 106'fc. 

T2b. 6{ekrj:,^j,^^^^2-W'^\>i. 

T2c. One of the following is true: either 

^ 2-1^(^)1 > {9k)'S,0T Y. 2l^Wn^M\-l-|A^('')l > ^/(0fc). 

bGNi{x,T[e]) bGN-^i{x,T[i]) 

T2d. X occurs in more than 100 clauses from T^ [£]. 

T2e. X occurs in a clause b that contains fewer than 3| A^(6) |/4 variables from H [£]. 



Proposition 14. 21 (and items QO and Ql from Definition 14. lb ensure that there are only a very few variables that 
satisfy HI, T2a, or T2b w.h.p. We always include these few into the set T2 [£ + 1] of 'exceptional' variables. 
Moreover, intuitively H2 and T2c-T2e capture variables x that are highly exposed to the 'exceptional' set T [£] 
from the previous round. Furthermore, we let 

Ts [£+l] = {ae^t: \N{a)\ > lOOfci A \N{a) \T[£]\< fci} \ T3 [£] 

contain all clauses that consist almost entirely of 'exceptional' variables from T [£], but without including the 
clauses from the previous set T3 [£]. Finally, 

T[£+l]=Ti[£+l]U T2 [£ + 1] U iV(r3 [£ + 1]). 

In Section lF2l we will verify that T [£] does indeed contain the set B [£] of biased variables. 

Proposition F.l We have B[£]<zT [£] for all £ > 0. 

Furthermore, in Section lF3] we will establish the following bound on the size of T [£\. 

Proposition F.2 We have \T[£]\< Q.WSnfor all £ > 0. 

Finally, in Section lF6l we will derive Theorem l4.3l from Proposition lF. H and Proposition lF.2l 



E2 Proof of Proposition lEll 

Lemma F.3 Let x be a variable and let b g N{x) be a clause. Let t^ = \N{h) C] B[P\\ x\. Then 

|1 - lib^AO m < 22-I^WI+*^ exp{6\N{b)\) forC^ 0, 1. 

Proof. We may assume that clause b is not a tautology, because otherwise /it^^(O) [i] ~ /.ib^xi^) [^] = 1- 
We consider two cases. The first case is that x appears negatively in b. Since for any y £ N{b) \ x we have 
fiy^bil) M = 5 + Ay^b [i] and /iy^&(0) [l] + My^t(l) [i] = 1, we see that 

..^.(i^^f^)M = i-sign(,,6)A,..[.]. 

Therefore, 



l-/ib^,(l)[£] = n I - sign {y,b) Ay ^f,[e] 



2 

yGN(b)\x 

= 2^-mb}\ Jl l-2sign{y,b)Ay^b[i] 

yeN(b)\x 

< 2i-I^WI.2*^. [] l + 2|A^^bM I < 1 + 21^1^^1+''' •(l+5)l^('')l 

< 2i-l^('')l+*''-cxp(|iV(6)|(5). 
Changing one sign in the above computation, we also see that 

M6^x(l)M < l + 22-I^WI+*^.exp(<5|7V(6)|). 
In the second case where x appears positively in b the computation is analogous. D 

Corollary F.4 Let x be a variable and let T C N{x) be a set of clauses. For each b € T let t^ = \br\ B [£] \x\. 
Assume that tf, < \N{b)\ — 2forall b £ T. Then /^(,->rc(0) [i] > Q for all b G T and 



Inff ^^b^^^l)[e_ 



,^^M6^.(0)[^] 



<^ 23-1^^1+*" exp(J|7V(6)|). 



beT 



Proof. The fact that ^b^^i^) [(] > for all & G T follows directly from the definition (|2]i of fib^x and the 
assumption that ti, < \N{b)\ — 2. Multiplying the terms from Lemma lR3] up for b G T,we see that 



b^T^^''^'^^'^^^ ber Uer 



^22-l^('')l+*^exp(,5|7V(6)|) 



A similar computation yields the lower bound. D 

Corollary F.5 Suppose that x (z H [i] is harmless and that a G N{x) is a clause such that \N{a) \T [I — 1]\ < 
fci. Then \Ax^a [(] \ < 0.01. 

Proof Condition H3 ensures that \N{b) \T [£-!]] > fci for any b G N{x) \ a, and thus pb^x{0) [i] > 0. 
Moreover, H2 yields 

y- 2\N(b)nTie^i]\x\~\N{b)\ < j_ 

bGNyi{x,T[e]} 

Since 6 < cxp{-cp), CorollarvlR4lappUed to T = A^>i(a; ^ a, £) = iV>i(x, T [£]) \ a impHes 

|P>l(x^a)[^]-l|<10-^ 
provided that p> po is sufficiently large. 



Furthermore, since by HI all clauses & € iV(a;) have lengths 0.16'fc < |A^(6)| < 109k,weha\e N<i{x,T [£ - 1]) D 
Ni{x,T[i- 1]). Therefore, 



\P<iix^a)[£]^l\ 



n 



/ib^,(i)[£-i] 



n 



/Z6^,(l)[£-l] 



beN<iix,T[e~l])\Ni{x,T[l~l]) ^ ^ '^ ' b&Ni{x,T[l-l]) ^ ^ '^ ' 



With respect to the second product, H2 and Lemma lRlJ vield for p > po large 



In 



n 



b<£Ni{x,T[e~l]} 



M6^,(0)[£-l] 



< 8 ^ 2-l^('')lcxp(<5|iV(6)|) 

b<£Ni{x,T[i-l]) 

< i6{ek)^s < lo-^ 



because |iV(6) I < lO^fcforall 6and(5 = cxp(-ce'fc). Furthermore, letting TV = Af<i(a;,r [£- l])\iVi(a;,r [£ - 1]), 
we obtain from HI and H4 



mn 



^lb^,il)[e^l] 



b&M 



^i^,^^(0)[i-l] 



< 



^sign(x,6)2i- 



\N{b)\ 



beAf 



cxp(106iM) 



< 4 



< 4 



^ sign(a:, 6)2" 



\N{b)\ 



beAf 



< 10 



^ sign(x,6)2-I^WI 

beN<i{x,Tie-i]) 

-6 



4 ^ 2-I^WI 

xeNi{x,Tli-l]) 



again for p > po large. Combining the two above estimate, we obtain |P<i (.x -^^ a) [^] — 1| = 3 • 10 ®. Finally, 
as pb^xiO) [f] > for all b G iV(2;) \ a, we see that 






= |1 - P<i (x ^ a) [i] ■ P>i (x ^ a) [£]| < 10"'' + 4 • lO"*' < 0.001. 

Since /ij.^a(l) [^] + M2-^-a(0) [i] = 1 by construction, this entails the desired bound on A^^a [£]■ 
Corollary F.6 Let bbea clause such that N{b) ^ T [P\. Let x e N{b). Then 



U 



Pb-^x{0) [^- 1] > and 



Pb^,{l)[e~l] ^ 

Pb^^io)[e-i] 



< cxp(-fci/2). 



Proof. We consider two cases. 

Case 1: \N{b) \ T [^ — 1] | > fci. Then the assertion is immediate from Lemma lF3] 

Case 2: \N{b) \ T [£ - 1]| < fci. Since N{b) (t T [£], condition T2a ensures that 0.16ifc < |A^(6)| < lOe'fc. The 
assumption N{b) ^ T [£] ensures that 6 ^ Ta [£]. Hence, the definition of T3 [£] entails that b € T^ll ~ 1]. 
As Nib) (/_ T2 [i], T2e impUes 

\N{b)nH[£^l]\>3\N{b)\/A. (38) 

Let J = N{b) nH[£- 1]. Since 6 e T3 [£ - 1], we have ^ > 2 and \N{b) \T[£~2]\ < ki. Therefore, 
Corollarv lF.5l implies that 

^ly^b{l)[£~l] ^ 



^iy^biO)[£-l] 

Consequently, (|38] | yields pb-*xiO) [£] > and 



< 0.1 for all 2/ e J. 



Pb^xii)[£-l] 



/ifa^,(0)[£-l] 



9\|./| /9\|A^('')l/4 

< i t) ^ ( r I < cxp (-6ifc/10^) 



Thus, we have established the assertion in either case. 



D 



Proof of Proposition \F.1\ We proceed by induction on L Since i? [0] = the assertion is trivial for £ = 0. Thus, 
assume that £ > and that B[i] CT [i]. Let x e Vt \ T [£ + 1]. We will prove x (^ B[i+1]. Corollary |R6 
implies that 



//a^,(0)[£] >Oand 



Ma^:r(l)| 



Ma^x(0)[£] 



-1 



< exp (-fci/2) for all x ^ T [£ + 1], a e N{x). 



We claim 



|P>i {x -^ a) [£ + 1] - 1| < 5/100 for all a; ^ T [£ + 1], a e N{x) 
To establish (l40l i. we consider two cases. 



(39) 



(40) 



Case 1: x ^ N{T3 [£]). Let T = A^>i {x — > a) [£ + 1] be the set of all clauses b that contribute to the product 
P>i (x ^ a)[£ + 1]. Since x ^ N{T3 [£] U T3 [£ + 1]), none of the clauses b € T features more than 
\N{b) I - fci variables from T [£] (just by the definition of T3 [£ + 1]). Furthermore, because x ^ T'2 [£ + 1], 
T2c is not satisfied and thus we obtain the bound 



ber beN>i{x,T[e]) 



(41) 



provided that Ok > p > po is sufficiently large. Therefore, (|40] | follows from Corollarv lF.4l 

Case 2: X e N{T3 [£]). Let T = A^>i (x -> a) [£ + 1] \ T3 [i] be the set of all clauses b that occur in the product 
P>i (.T -^a)[i + l], apart from the ones in T3 [£]. Since 2; ^ Ta [£ + 1] U 7V(r3 [£ + 1]), this set T also 
satisfies (HTI) . Thus, Corollarv lF.4l vields 



1 n 



/ib^,(l)[£+l] 



hGT 



/i6^.(0)[£+l] 



< (5/10^ 



(42) 



As condition T2d in the definition of T2 [i + 1] ensures that \N^^^ [£ + 1] n T3 [£] \ < 100, ^ and (|42li 
imply 



\1-P>i{x^a)[i+1]\ = 



f^b~,x{l)[i] 






n 



M6^.(l) [£] 



= (5/10- 



1^ c, 3 

< 5/10^3 + 2 



1- 



n 



Aifc^2;(l) 



bew^^^[£+i]nT3[, 



Mfc^.(O) 



1- ]J 1 + cxp (-6'fc/105) 

beNi^^{i+\\r\T^{e\ 



< J/500. 



which establishes (|40] |. 



If .T ^ Ti [£ + 1] C T [£ + 1] and a e iV(a;), then |P<i (.t ^ a) [£ + 1] - 1| < ^100. Hence, gOll implies 
that for all X ^ T [£ + 1] and all a £ N{x) we have p-x^aiO) [£ + 1] > and 



1- 



Ai,^,(l)[£+1] 



M,_,a(0)[£+1] 
Consequently, {A^^a [^ + 1] I < 0.1(5, and thus x ^ B [£ + 1] 



= |l-P<i(x^a)[£+l]-P>i(x^a)[£+l]| < (5/99. 



D 



E3 Proof of Proposition lE2l 

Lemma F.7 Assume that |Ti [i] U T2 [£]\ < 9Sn/3 and \N{T:i [l])\ < e5n/2. Then \N{T3 [( + 1])| < e5n/2. 

Proof. By construction we have T3 [£] n Ts [^ + 1] = 0. Furthermore, also by construction N{T3 [£]) C T [i], and 
each clause in T3 [£ + 1] has at least a 0.99-fraction of its variables in T [£]. Hence, Q3 yields 

\N{n[£+i])\ + \NiT,m < J2 l^wi [] 

bBT3[i]UT3[l+l] 

< im\T[£] I < 1.03 (iTi [e]\ + IT2 [i]\ + \N{T3 m) . 

Hence, |iV(T3[^+l])| < 1.03 (|Ti [£]| + jTs Ml) + 0.03 |7V(T3 [^])| < eSn/2. D 

LemmaF.8 Assume that \Ti [£] U T2 [£]\ < 05n/3and\N{T3 [£])| < 05n/2. Moreover, suppose that \T[£ - l\\ < 
eSn. Then jTa [^ + 1]| < 9Sn/6. 

Proof. Conditions QO and Ql readily imply that the number of variables that satisfy either T2a or T2b is < 
O.QQ196n. Moreover, we apply Q2 to the set T [£] of size 

\T [£]\ < |Ti [£] U T2 [£]\ + \N{n m < O.mn (43) 

to conclude that the number of variables satisfying T2c is < O.QQWSn as well. 

To bound the number of variables that satisfy T2d, consider the subgraph of the factor graph induced on 

Ta [£] UN{T3[£]). For each x e N{T3 [£] ) let D.^ be the number of neighbors of x in T3 [£] . Let 1/ be the set of all 
X eVtSO that Dx>2. Then Q3 yields 

imu < J2 ^^"^ Yl l^(a)l < 1-01|T[£] I <6l(5n, 

because N{b) C T [£] for all b € T3 [£]. Hence, there are at most 1/ < O.OWdn variables that satisfy T2d. In 
summary, we have shown that 

\{x <eV -.x satisfies one of T2a-T2d}| < Q.Ol505n. (44) 

To deal with T2e, observe that if a clause a has at least | A^(a) |/4 variables that are not harmless, then one of 
the following statements is true. 

1. a contains at least \N{a)\/2Q variables x that violate either HI, H2, or H4. 

2. a contains at least | A^ (a) |/5 variables x that violate condition H3. 

Let Ci be the set of clauses a for which 1. holds, and let C2 be the set of clauses satisfying 2., so that the number 
of variables satisfying T2e is bounded by J2aeCiUC2 \^(^)\- 

To bound Ci, let Q be the set of all variables x that violate either HI, H2, or H4 at time £. Then conditions 
Ql and Q2 entail that |Q| < 2 • lO^^'^^iJn (because we are assuming \T{£ — 1)| < 66n). Therefore, condition Q3 
implies that 

Y^ \N{a)\ < 21 \Q\ < O.OOSeSn. (45) 

aeCi 

To deal with C2 let B be the set of all clauses b such that \N{b)\ > lOOfci but \N{b) \T[£]\ < ki. Since we 
know from (|43]i that \T [£] \ < BSn, condition Q3 appHed to T [£] implies that Y,b(zB 1^ W I < 1-03 \T [£]\. Thus, 

\NiB)\ < Y^ \N{b)\ < 1.03 \T [£]\ < l.OWSn. (46) 

beB 

Furthermore, let U be the set of all clauses a such that N{a) C N{B). Let U be the set of variables x E N{B) 
that occur in at least two clauses from U. Then by Q3 



\U\ + \NiB)\ < ^|iV(a)|<1.01|iV(e)|, 



aeu 



whence \U\ < 0.01\N{B)\ < 0.0295n. Since B C 14, the set U contains all variables that occur in at least two 
clauses from B, i.e., all variables that violate condition H3. Therefore, any a E C2 contains at least |A^(a)|/5 
variables from U. Applying Q3 once more, we obtain 

1^(^2)1 < ^ |iV(a)| < 5.05-0.026'(5n = 0.126'(5n. 

aGC2 

Combining this estimate with the bound (l45T l on Ci, we conclude that the number of variables satisfying T2e is 
bounded by J^aec uc l^l*^)! — 0.1236'(5n. Together with (l44l i this yields the assertion. 
In Section lF!4l we will derive the following bound on \Ti{(. + 1)|. 

Proposition F.9 If\T[e]\< 9Sn, then |ri(£ + 1) \ r2(£ + 1)| < e5n/Q. 



(47) 



D 



Proof. rProposition lF.2l We are going to show that 

|Ti [(] U T2 [e\\ < e5n/i and |iV(r3 [^])| < e5n/2 



for all ^ > 0. This implies that |r [^]| < B5n for all £ > 0, as desired. 

In order to prove Wi\ we proceed by induction on t. The bounds for £ = are direct from QO and Ql. Now 
assume that (gTli holds for all I < £. Then Lemma |R7] shows that |iV(T3 [i])\ < eSn/2. Moreover, Lemma |R8] 
appHes, giving [Ta [i + l]\ < 9dn/6. Finally, Proposition |R9] shows \Ti{£ + 1) \ T2ii + 1)| < edn/6, whence 

|Ti [^ + 1] U T2 [£ + 1] I < eSn/3. a 



F.4 Proof of Proposition lE9l 

Throughout this section we assume that \T[I[\\ < 65n. 
For a variable x £ Vt and a £ N{x) we let 

a,^a[i + l] = Y. 2i-I^WIsign(x,6), 

beN<i(x^a)le+l] 

i.^a[e + l] = E E 2i-I^WIsign(z,6)sigii(y,fo)A,^,[£], 

bi£N<i{x^a)[l+l] yeN{b)\x 

and Lx^a [^ + 1] = o^x^a [^ + 1] + Cx^a [^ + 1] • In Section lRSJ we are going to establish the following. 
Proposition F.IO For any variable x ^ T2[t + 1] and any clause a £ N{x) we have 

Lemma F.ll For all but at most Q.QlOSn variables x £ V \T2[i + I] we have 

max |cr^_><j[£ + l]| < 0.001(5. 

aeN{x) 

Proof. This is a direct consequence of condition Q2. D 

Lemma F.12 Let x be a variable and let 61, 62 £ N{x) be such that \N{hi) r\T[P\\ < 2 and \N{hi)\ > 0.19k 
fori^ 1,2. Then |A,^b, [i] - A^^b, Ml < S\ 

Proof. Our assumptions ensure that N{bi) \ T 7^ and thus i^x^bt (0) [i] > for i = 1,2. Corollarv lF.6l vields 

fib,^,{l)[£^l] 



fori = 1, 2. Let qi 



fib,^x{0)[e- 1] >Oand 



Mh.-.x(0)[^-l] 



< cxp(-fci/2) 



(48) 



Mx-+6i(0)[. 



Then 



^x^b, [i]\ 



|/i.^bi(l)M-Mx^62(l)l 



91 -92 



< 



1- 



92 

91 



(l + 9i)(l + 92) 

/ibi^.(o)[£-i] fib,^^{i)[e-i] 

Hence, the assertion follows from (l48l l. provided that the constant c in the definition ^ of 5 = St is chosen 
sufficiently small. D 



Lemma F.13 For all but at most Q.W5n variables x ^ T2[£ + 1] we have max^ig^vfa;) l^a;->a| < 0.001(5. 

Proof. For a variable y let /^{y) be the set of all clauses b E N{y) such that b £ N<:i{x, T [£]) for some variable 
X € V. If M{y) = we define Ay = 0; otherwise select Uy G J^{y) arbitrarily and set Aj, = A^^a [£]. Thus, 
we obtain a vector A — {Ay)y^y with norm || A||^ < i. Let S = {£,x)xev ~ ^T[e]^^ where Ax[£] is the Unear 
operator from condition Q4 in Definition |4T| That is, for any x G V we have 



^-= E E 2i-I^WIsign(x,&)sign(2/,6)A,. 

bl^N<i{x,T[t]) yeN{b)\x 

Because \T[£]\ < 6'fe, condition Q4 ensures that HAtj^jHi-i < 6^9n. Consequently, 

\\E\\, = ||Aj,[,]A||^ < ||Aj,[,]||^ ||A||^ < S^0n. (50) 

Since ||S|| ^ = X^i^ev \ix\, (ESll implies that 

\{xeV :\^x\>S^}\<S^0n. (51) 

To infer the lemma from (ISTT i. we need to establish a relation between £^x and ^x^a for x £ V\T2[£ + I] and 
a S N{x). Applying Lemma IF. 121 we get 

\Ay^b M - Ay^b' [i] I < S^ for all yeV,b,b' e AA(y). (52) 

Hence, 

|Aj,^fc[^]-Aj,|<,52. (53) 

Consequently, we obtain for all x ^ T2[£ + 1] 

max |^.^.-C.| < ^ 2i-I^Wl|A,|+ ^ ^ 2i-I^WI |A,^4^] - A,| 

aGA/fa;) — — — 

y&N{a) beN<i(x^a)[i+l]yeN{b)\x 

f |iV(a)|2-l^('')l+2,53 ^ |iV(5)|2i-I^WI 

fc6W<i(2:-i.a)[£+l] 

< 10-^(5 [byQ2], (54) 

provided that Ok > p> pais sufficiently large. If a; e V \ T2 [£ + 1] is such that j^^^l < S'^, then ( |54] | implies that 
I^K^al < 2 • 10^*6 for any a g N{x). Therefore, the assertion follows from dSTI ). D 

Proof. [Proposition |F.9l Let 5 be the set of all variables x ^ T2 [£ + 1] such that max^g^j^.) |cr2:_i.Q| < 0.001(5 
and m.a.XaeN{i) \£,x^a\ < 0.001(5. Then Proposition IF. 101 entails that for any x £ S and a £ N{x) 

|lnP<i {x^a)[e+l]\< \Lx^a [e+l]\ + 10-3(5 = a,^, + C.^, + 10-^^ < 0.003(5. 

Hence, |P<i (a; ^ a) - 1| < 0.01(5 for ailx £ S,ae N{x), and therefore 

Ti [^ + 1] \ T2 [^ + 1] c 1/ \ (5 u T2 [e + 1])). 

FinaUy, Lemmas Em and EB] imply |Ti [^ + 1] \ T2 [£ + 1]| < |y \ (S" U T2 [£ + 1])| < 6l(5n/6. D 

F.5 Proof of Proposition lElOl 

Lemma F.14 Let x be a variable, a £ N{x), andb £ N<i {x — > a) [£ + 1]. Then fib-^xiO) [i] > and 



1„ f lib^x{l)[i] \ _ _r,l-\N{b) 



sign (cc, 6) + 2 ^ sign (x, 6) sign (y, 6) A,y_>6 [£] 

yeN(b)\x 

(ekd + \N{b)nT[i]\x\) 



(55) 



2l^('')l 
where the O {■) denotes an asymptotic bound for p sufficiently large. 



0{ek5), 



Proof. If & e N<i [x -^ a)[i+ 1], then \N{h) n T [^] | < 2, while \N{h)\ > O.Wk. Therefore, LemmalR3]shows 
that |-i — jib^xi^)) [l] \ < P, provided that the constant c in the definition (|6]l of (5 is chosen sufficiently small. 
Furthermore, b is not redundant, and thus not a tautology, because otherwise N{h) C T2 [(] due to T2a. In order 
to establish (ISST i we consider two cases. The first case is that sign (cc, fe) = — 1. Then /ib_>^(0) [£] = 1 and thus by 
definition ^ 

^^^=i^M ^ ;.,^,(l)[^] = l- n /i,^,((l-sign(y,6))/2)[£] 
= 1- n l-sign{y,b)Ay^,[e] 

yeN(b)\x 

^ l_2i-I^WI n l-2sign(y,6)A,^,[£]. (56) 

y£N{b)\x 

Let r = N{b) \ {T [i] U x). As Proposition IrT] shows that T [£] D B [£] contains all biased variables, we have 
\Ay^b [^] I < (5 for all y G r. Therefore, we can use the approximation |ln(l — z) + z\ < z^ for \z\ < 5 to obtain 



In Jl 1 - 2sign (y, b) Ay_ 
yer 



^J22sign{y,b)Ay^b[i] 
yer 

< 4^Ay^b[if <A09k5'^; 
yer 



(57) 



in the last step, we used that |iV(6)| < lO^fc for all b E N<i {x — > a) [^ + 1]. Furthermore, by the definition of 

iV<i {x -^ a), we have \N{b) nT[e]\x\< 1. If there is yo(E N{b) n T [£] \ x, then dSTll yields 

n 1 - 2sign (y, b) A,_b [£] = (1 - 2sign (yo, b) A^„_b [£]) • J] 1 - 2sign (y, b) A,^, [i] 

yeN{b)\x yer 



1-2 



J2 signiy,b)Ay_,b[e] +0{ek5f 

yeN(b)\x 

-4sign{yo,b)Ay„^b[£] ^ sign(y, &) Aj,_ 



= 1-2 



Y^ sign {y, b) Ay^t \ 

yeN{b)\x 



yeN{b)\x 

o{ekS), 



where the O (•) hides a term that remains bounded independently of p. Hence, in any case we have 



Yl l-2sign{y,b)Ay^i,[e] = 1-2 

yeN{b)\x 



Y^ sign (y, 6) Ay^t | 

yeN{b)\x 



+ {0kS + \N{b) r\T[£]\ x\) ■ O{0kd). 
Combining this with (l56b and using the approximation |ln(l — z) + z\ < z'^ for \z\ < 1/2, we see that 

'tib^x{i)[eV 



In 



^l-b^x{0) 



= -2 



1-|W(6)| 



II l-2signiy,b)Ay^b[i] 

yeNib)\x 



+ 0(22-2|^WI) 



^2i-\N{b) 



sign (a:, 6) +2 ^ sign (x, 6) sign(y, 6) Aj^_ 

yeN{b)\x 

+2^-\N(^)\{0kS + \N{b) nT[e]\ x\) ■ o{ek5). 



In the case sign (a;, a) = 1 the proof is similar 



D 



Proo/; rProposition lF. ion Lemma lF. 14l entails that for large p 

>b^.(l)[^] 



lnP<i(a;^a)[£+l] 



E 



In 



M6^.(0)M 



Y, -Lb^x [^ + 1] + 2-l^^'')l(0fc(5 + \N{b) r\T[(.]\ x\)0{ek5) 

beN<i{x^a)[e+i] beN°^je+i] 

+o{ek5) Y 2-i^wi 

be Wi {x-^a) [i+l] nN< i (x^a) [l+l] 

0{6k5f 



J2 Lb^x [e + i] + 



{ek)36 



0{ek5) ■ o{pS) 



beN<i{x^a)[i+l] 

= - Y Lf,^x[e + l] + 0{S), 

b£N^i{x^a)li+l] 

due to conditions T2b and T2c. Hence, there is po > such that for p > p^we have 



Y Lb^x [i+l]+ In P<i{x^ a) [i + 1] 

b£N<i{x^a)l£+l] 



< 10-^6, 



as desired. 



D 



F.6 Completing the proof of Theorem 14.31 

We are going to show that \pxi^t-i,i^) ~ ^\ ^ S = 5t for a\\ x € Vt \ T [uj + 1]. This will imply Theorem l4.3l 
because jT [cj + 1] | < St{n — t) by Proposition |R2l 

Thus, Ittx eVt\T[u} + l]. Corollary |E6] shows that pb^xiC) [^] > for C = 0, 1. Hence, 

^(1) = n ^'b^A^) H > 0, and P(0) = Yl pb^xiO) H > 0. 

bGN(x) b£N{x) 

Recall from (HI that 

P(l) 

^^ ' ' P(0) + P(1) 

If A^(2;) = 0, then trivially P(0) = P(l) = 1 and thus /i^($t_i,a;) = i. Thus, assume that iV(x) ^0andpick 
an arbitrary a £ N{x). Then by Q 

P(C) = Pa-.xiOH-l^x^aiC)[uJ + l] for C = 0,1. 

Since x ^ T [ui + 1] D B[uj + 1] (by Proposition lF. lb . we have 



Mx^a(C) [w + 1] - 2 



< 0.16 



for C = 0, 1. Therefore, assuming that 9k > p > po is sufficiently large and thus S = cxp{—cdk) is sufficiently 
small, we obtain 

1 M.^a(O) [^ + 1] ^ , 1+0-2'^ ^n.X H 1 1 

^ %.^Jl)[c. + l] ^ ^"13^^"-''^' -d analogously 
ln ^-"S°;i" + ^! > -0.56. 
Furthermore, Corollary |F6] yields 

Ata^T(O) N 
^a^i;(l) H 



<2cxp(-6'fc/10''^) <6'\ 



Hence, 



In 



P(0) 



P(l) 



< 



hi 



^ix^a{0) [w + 1] 



Therefore, letting z = In -phr, we obtain 



- Hxi^t-l,^^] 



P(l) 



P(0) + P(1) 



as desired. 



In 



^J■a^xio) H 



^a^:E(l)H 



<0.55 + S^ < 0.515. 



1 



2 1 + cxp(z)/2 



< exp(z) < 0.51(5, 



G Proof of Corollary [TJ 



Let us begin by disproving the second statement of Hypothesis 11.31 Suppose that r is as in ([T]l- Let us call a 
formula <1> (w, e)-simple if 

P [Vt < n : |A4,($t) - /i=..(*t,w)l <£]>!- o(l), (58) 

where, of course, probability is taken over the execution of Experiment ! 1.2l onlv. Then Hvpothesis ll.3l claims that 
for any e > there exist a function oj = 0Je{n) and a number ?; = ri{e, k, r) such that the random formula $ is 
(w, e)-simple with probability at least rj. (Actually, Hvpothesis ll.Sl claims that this statement is true for any fixed 
77 < 1.) Assume for contradiction that this statement is true. 

Let (Tidoai be the (random) assignment generated by Experiment ! 1.21 and let ctbp be the assignment constructed 
by BPdec on the same random input $. Furthermore, let Q = Gi^) be the set of all satisfying assignments a 
such that given (Tidcai = fr it is true that \Mxt{^t-i) — l^xt{^t-i,^)\ < £ w.h.p. for all 1 < t < n. Then, by 
assumption ( fSSl ), for a random formula 4> we have \Q\ = (1 — o(l))2" with probability at least rj > 0. 

Let $ be a formula such that indeed |5($)| = (1 - o(l))2" w.h.p., and let a G G{^)- Let 8t be the event 
that there is an cr G t? such that cridcai(a^s) = crBp(a^s) = cr{xs) for all 1 < s < i. Given that £t-i occurs, 
we have \Mxt{^t-i) — IJ-xt{^t~i,^)\ < £■ Hence, by Steps 3 of Experiment [TT2l and BPdec and because 
\Q\ = (1 — o(l))2" we have P [£(|£t_i] > 1 — £. Multiplying these terms up, we see that 



P [£„] > cxp(-£n) 



(59) 



But if £n occurs, then the output ctbp is a satisfying assignment. Hence, Theorem 11.11 implies that there is an 
£0 = eo(fcj r) > (namely, the constant hidden in the J7 (•) in Theorem 1 1.1b such that ( |59] l is false for e < Eq, in 
contradiction to part ii. of Hypothesis [T3] 

We come to the first statement. Assume for contradiction that there is a function 7 = 7(71) — > 00 so that with 
probability 1 — exp(— 7) the random formula $ is such that for the sequence #t generated by Experiment ll .2l we 
have 

P [Vi < 71 : |Af,,(*t) - A4,(#t,c^)| < e] > 1 - cxp(-7) (60) 

for any e > and u = uj{e, k,r) > 1 sufficiently large. 

Lemma G.l There is a number r/ = rj{e, k,r) > such that with probability > rj the factor graph of $ has no 
cycle of length < 2uj. 



Lemma IGTI is a standard result in the study of random hypergraphs. It follows easily from the fact that the 
total number of cycles of length < 2uj converges in distribution to a Poisson distribution with mean exp(Caj) for 
a certain C = C{k, r) > iflTl Section 9.2]. Therefore, the probability that there is no such cycle converges to 
r]{e, k, r) = exp(— exp(Cw)) > 0. 

Let A be the event that the factor graph of $ has no cycle of length < uj. Then ( |60] | entails that for large 
enough n the formula $ satisfies (I6OI 1 with probability at least 1 — cxp(— 7)/77 = 1 — o(l) even if we condition on 
the event A. Furthermore, if the event A occurs, then Mx^ ($t, cj) = jixt (^t-i j ^) because Belief Propagation 
yields the correct marginals if there is no cycle of length < 2aj. Thus, we conclude that the with probability at 
least rj — o(l) the random formula $ is such that 



P [Vi < n : \MxA^t) - A^x,(*t,^)| <£]>!- o(l), 



(61) 



which is precisely the statement that we refuted in the first part of the proof. 



